kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #4396] [RFC]: [Store] Client Configuration Normalization and Validation Pipeline #1107

New issue

Open

opened 2026-10-29 21:22:57 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:22:57 +00:00

Owner

Originally created by @Aionw on GitHub (Sep 29, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4396

Changes proposed

Context and goal

A Mooncake Store client is configured through several unrelated input paths, and each path resolves, defaults and validates settings on its own.

Entry point	Inputs
`MooncakeDistributedStore.setup(...)` positional args (`store_py.cpp`)	explicit args; `ipc_socket_path`, local RPC port `50052` and `start_offload_rpc_server=true` hard-coded downstream
`setup(dict)` (`ConfigDict`)	string map parsed by `get_config*` helpers in `real_client.cpp`
`init_all(...)` / `mooncake_store_init_all`	three args; master `localhost:12345`, metadata `127.0.0.1:2379` and a 1 GiB buffer hard-coded
C API `mooncake_store_setup`	seven args
`mooncake_client` binary (`real_client_main.cpp`)	gflags, including `--enable_http_server` / `--http_port` defined in `real_client.cpp`, which therefore also apply in-process for pybind
Python `MooncakeConfig` (`MOONCAKE_CONFIG_PATH` JSON or `MOONCAKE_*` env)	converted to the dict path

On top of that, components below RealClient read their own settings directly from the process environment at construction, at first use, or on every call (Environ::Process(), ~50 production call sites after #4388).

This causes three problems:

Divergent defaults and precedence. The same setting resolves differently per entry point. Only the dict path checks segment sizes; the offload RPC port is 50052 everywhere except the binary.
Late validation. Several invalid inputs are reported only after side effects have happened:
- a missing MC_CXL_DEV_SIZE is fatal only after the client is created and its buffer registered;
- the HTTP port range is checked after the client, segments, IPC server and offload RPC server are running;
- FileStorageConfig and the storage backend configs throw only while offload initializes.
No single place answers "what does this client run with?" Values are resolved inside components, so they can't be logged, tested or injected as a whole.

Goal: normalize every external client input into one typed ClientConfig, validate it once before any component is created, and construct the client from it. Components will then receive their part of the config from their parent instead of reading external sources.

This follows the direction of #3809 (environment convergence) and #4360 (component-owned configuration, resolved once at startup) for the client side. #4360 covers the master.

Pipeline

external inputs                      normalize            validate          construct
─────────────────────────────────    ───────────────     ────────────      ─────────────────────
python args / dict / MooncakeConfig ─┐
C API args                          ─┤  ClientConfig      ClientConfig::    RealClient::Setup(
mooncake_client gflags              ─┼─► (one struct)  ─►  Validate()    ─►   const ClientConfig&)
process environment (Environ)       ─┘                                      └► Client / sub-components

Normalize. Each entry point has a small adapter that converts its own input into ClientConfig: FromSetupArgs, FromConfigDict, FromCommandLine, and the C API. Adapters only translate; they don't validate or start anything. Environment-backed sub-configs come from the existing XxxConfig::FromEnvironment(Environ::Process()) loaders, using the per-process catalog ClientEnvironmentVariables (#4392).
Validate. ClientConfig::Validate() runs every check in one place: per-field checks, cross-field checks such as "protocol cxl requires a CXL device size", and the existing Validate() of each sub-config. It returns an error before anything is created. Every entry point goes through it.
Construct. RealClient::Setup(const ClientConfig&) becomes the single setup path. The existing positional, dict, init_all, C API and binary entry points become thin wrappers that normalize, validate and then call it.
Distribute (follow-up PRs). RealClient and Client pass sub-configs to the components they create, so the remaining Environ::Process() reads under the client are removed one component subtree at a time.

`ClientConfig` shape

The top-level groups mirror ClientEnvironmentVariables from #4392, so each setting has one owner group:

ClientConfig
├─ identity        local_hostname, host_id, tenant_id, labels
├─ master          master_server_entry, metadata_server; rpc {protocol, timeouts, io_threads}
├─ transport       protocol, device_names, auto_discovery {enabled, filters}, auto_port, numa
├─ memory          global_segment_size, local_buffer_size, hugepage, mmap_arena,
│                  pinned_memory, cxl_segment, shm_spdk_registration
├─ metrics         ClientMetricConfig, http_server {enabled, port}
├─ transfer        submitter (memcpy override), fileread_workers, nof {workers, debug, register}
├─ local_hot_cache, replica_selection, object_checksum
└─ offload         enabled, start_rpc_server, local_rpc_port,
                   FileStorageConfig + backend: Bucket | FilePerKey | OffsetAllocator |
                   NvmeKv {connector, executor, io_concurrency} | Distributed {DFS, OSS}

Sub-configs reuse the existing typed config structs under mooncake-store/src/config, so no new parsing logic is introduced.

Precedence contract

This RFC does not change precedence. Where more than one source feeds a setting, the adapter reproduces the current effective order, and a test pins it. The known overlaps are:

Setting	Current precedence (kept)
SSD offload path	non-empty explicit arg > `MOONCAKE_OFFLOAD_FILE_STORAGE_PATH` > default
Client HTTP server	enabled = arg OR `--enable_http_server`; port = arg if arg-enabled, else `--http_port`
RDMA devices / auto-discovery	`MC_MS_AUTO_DISC` > (protocol rdma/efa and no device → on); filters only when on
Host id	`MOONCAKE_HOST_ID` > normalized `local_hostname`
Client port	explicit port in `local_hostname` disables the auto-port range
Local offload RPC port	binary `--port`; other entry points `50052` (now an explicit `ClientConfig` default)
mmap arena	`MC_DISABLE_MMAP_ARENA` wins; enabled = flag OR pool-size env; env size > flag size
Python `MooncakeConfig`	service CLI dict > (JSON file if `MOONCAKE_CONFIG_PATH`, else env)

Any change to precedence or defaults requires a separate proposal, consistent with #3809 and #4360.

Behavior changes

The only intended changes are in when validation happens and which entry points run it. They are listed in each PR description:

Earlier failure. Invalid inputs now fail before the client connects to the master or registers memory:
- missing or invalid CXL size;
- out-of-range HTTP port;
- invalid offload / storage-backend config.
The conditions and error codes stay the same. Only the point of failure moves earlier.
Uniform checks. Segment-size limits, which today only the dict path enforces, apply to every entry point.

Out of scope

Master configuration. It follows #4360 (ResolveXConfig owners, default → file → explicit CLI).
Dummy client. It has no Client and talks to the real client over RPC; it can adopt the same pattern later.
Runtime settings the master sends over RPC (fsdir, disk eviction, quota). They aren't startup inputs and stay where they are.
Hot reload, a public grouped config-file schema, and renaming flags, keys or environment variables.

Process-wide state

Several client settings are consumed by process-wide singletons or caches:

ShmHelper, SpdkWrapper and the registered pinned-memory manager;
the call_once mmap arena;
function-local statics for fileread / NoF worker pools, replica selection, object checksum, and RPC client I/O threads.

Because of these, two RealClient instances in one process already cannot use different values for those settings. ClientConfig keeps them in their groups, but the follow-up distribution PRs mark them process-scoped. Those PRs initialize them once from the first validated config and log a warning if a later config differs, rather than silently ignoring it. Moving them to per-instance ownership is out of scope.

Delivery plan

Each step is its own PR, stacked on the previous one, with no behavior change beyond the validation timing listed above.

Entry normalization and validation (client). This step:
- adds ClientConfig with Validate();
- adds the entry adapters and RealClient::Setup(const ClientConfig&);
- routes all five real-client entry points through it;
- moves the early checks up front and adds tests for every precedence row above.
Components are not changed yet. Because Environ::Process() snapshots the environment on first read (#4388), the values validated at the entry are the same values the components read later.
Distribute: offload subtree. FileStorage and CreateStorageBackend receive offload instead of reading Bucket / FilePerKey / OffsetAllocator / DFS / OSS / NVMe-KV environment settings. This also removes the per-call NVMe-KV reads.
Distribute: transfer subtree. TransferSubmitter, the worker pools and NoF.
Distribute: memory and master-connection subtrees.
- Memory: hugepage and mmap arena, including the alloc/free re-reads.
- Master connection: MasterClient pool config and replica selection. These are currently read inline in headers.
- Remaining RealClient / Client reads.

When step 4 is done, no component under the client reads external configuration directly.

Relationship to existing work

#3809: environment convergence (Phase 1) and the catalog rule. This RFC builds on the typed, injectable environment reader in #4388 and the per-process catalog in #4392.
#4360: the same ownership principle on the master side. The two RFCs agree on:
- no precedence or default changes;
- one resolved value per setting;
- business code receiving typed configs instead of FLAGS_* or environment lookups.

Open questions

Should ClientConfig live under mooncake-store/include/config (public to pybind and the C API) or be split into a public input struct and a private resolved struct?
When a process-scoped setting differs between two clients in the same process, should that be a warning, as proposed, or a hard error?

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @Aionw on GitHub (Sep 29, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4396 ## Changes proposed ### Context and goal A Mooncake Store client is configured through several unrelated input paths, and each path resolves, defaults and validates settings on its own. | Entry point | Inputs | |---|---| | `MooncakeDistributedStore.setup(...)` positional args (`store_py.cpp`) | explicit args; `ipc_socket_path`, local RPC port `50052` and `start_offload_rpc_server=true` hard-coded downstream | | `setup(dict)` (`ConfigDict`) | string map parsed by `get_config*` helpers in `real_client.cpp` | | `init_all(...)` / `mooncake_store_init_all` | three args; master `localhost:12345`, metadata `127.0.0.1:2379` and a 1 GiB buffer hard-coded | | C API `mooncake_store_setup` | seven args | | `mooncake_client` binary (`real_client_main.cpp`) | gflags, including `--enable_http_server` / `--http_port` defined in `real_client.cpp`, which therefore also apply in-process for pybind | | Python `MooncakeConfig` (`MOONCAKE_CONFIG_PATH` JSON or `MOONCAKE_*` env) | converted to the dict path | On top of that, components below `RealClient` read their own settings directly from the process environment at construction, at first use, or on every call (`Environ::Process()`, ~50 production call sites after #4388). This causes three problems: - **Divergent defaults and precedence.** The same setting resolves differently per entry point. Only the dict path checks segment sizes; the offload RPC port is `50052` everywhere except the binary. - **Late validation.** Several invalid inputs are reported only after side effects have happened: - a missing `MC_CXL_DEV_SIZE` is fatal only after the client is created and its buffer registered; - the HTTP port range is checked after the client, segments, IPC server and offload RPC server are running; - `FileStorageConfig` and the storage backend configs throw only while offload initializes. - **No single place answers "what does this client run with?"** Values are resolved inside components, so they can't be logged, tested or injected as a whole. **Goal:** normalize every external client input into one typed `ClientConfig`, validate it once before any component is created, and construct the client from it. Components will then receive their part of the config from their parent instead of reading external sources. This follows the direction of #3809 (environment convergence) and #4360 (component-owned configuration, resolved once at startup) for the client side. #4360 covers the master. ### Pipeline ```text external inputs normalize validate construct ───────────────────────────────── ─────────────── ──────────── ───────────────────── python args / dict / MooncakeConfig ─┐ C API args ─┤ ClientConfig ClientConfig:: RealClient::Setup( mooncake_client gflags ─┼─► (one struct) ─► Validate() ─► const ClientConfig&) process environment (Environ) ─┘ └► Client / sub-components ``` 1. **Normalize.** Each entry point has a small adapter that converts its own input into `ClientConfig`: `FromSetupArgs`, `FromConfigDict`, `FromCommandLine`, and the C API. Adapters only translate; they don't validate or start anything. Environment-backed sub-configs come from the existing `XxxConfig::FromEnvironment(Environ::Process())` loaders, using the per-process catalog `ClientEnvironmentVariables` (#4392). 2. **Validate.** `ClientConfig::Validate()` runs every check in one place: per-field checks, cross-field checks such as "protocol `cxl` requires a CXL device size", and the existing `Validate()` of each sub-config. It returns an error before anything is created. Every entry point goes through it. 3. **Construct.** `RealClient::Setup(const ClientConfig&)` becomes the single setup path. The existing positional, dict, `init_all`, C API and binary entry points become thin wrappers that normalize, validate and then call it. 4. **Distribute** (follow-up PRs). `RealClient` and `Client` pass sub-configs to the components they create, so the remaining `Environ::Process()` reads under the client are removed one component subtree at a time. ### `ClientConfig` shape The top-level groups mirror `ClientEnvironmentVariables` from #4392, so each setting has one owner group: ```text ClientConfig ├─ identity local_hostname, host_id, tenant_id, labels ├─ master master_server_entry, metadata_server; rpc {protocol, timeouts, io_threads} ├─ transport protocol, device_names, auto_discovery {enabled, filters}, auto_port, numa ├─ memory global_segment_size, local_buffer_size, hugepage, mmap_arena, │ pinned_memory, cxl_segment, shm_spdk_registration ├─ metrics ClientMetricConfig, http_server {enabled, port} ├─ transfer submitter (memcpy override), fileread_workers, nof {workers, debug, register} ├─ local_hot_cache, replica_selection, object_checksum └─ offload enabled, start_rpc_server, local_rpc_port, FileStorageConfig + backend: Bucket | FilePerKey | OffsetAllocator | NvmeKv {connector, executor, io_concurrency} | Distributed {DFS, OSS} ``` Sub-configs reuse the existing typed config structs under `mooncake-store/src/config`, so no new parsing logic is introduced. ### Precedence contract This RFC does **not** change precedence. Where more than one source feeds a setting, the adapter reproduces the current effective order, and a test pins it. The known overlaps are: | Setting | Current precedence (kept) | |---|---| | SSD offload path | non-empty explicit arg > `MOONCAKE_OFFLOAD_FILE_STORAGE_PATH` > default | | Client HTTP server | enabled = arg OR `--enable_http_server`; port = arg if arg-enabled, else `--http_port` | | RDMA devices / auto-discovery | `MC_MS_AUTO_DISC` > (protocol rdma/efa and no device → on); filters only when on | | Host id | `MOONCAKE_HOST_ID` > normalized `local_hostname` | | Client port | explicit port in `local_hostname` disables the auto-port range | | Local offload RPC port | binary `--port`; other entry points `50052` (now an explicit `ClientConfig` default) | | mmap arena | `MC_DISABLE_MMAP_ARENA` wins; enabled = flag OR pool-size env; env size > flag size | | Python `MooncakeConfig` | service CLI dict > (JSON file if `MOONCAKE_CONFIG_PATH`, else env) | Any change to precedence or defaults requires a separate proposal, consistent with #3809 and #4360. ### Behavior changes The only intended changes are in **when** validation happens and **which entry points** run it. They are listed in each PR description: - **Earlier failure.** Invalid inputs now fail before the client connects to the master or registers memory: - missing or invalid CXL size; - out-of-range HTTP port; - invalid offload / storage-backend config. The conditions and error codes stay the same. Only the point of failure moves earlier. - **Uniform checks.** Segment-size limits, which today only the dict path enforces, apply to every entry point. ### Out of scope - **Master configuration.** It follows #4360 (`ResolveXConfig` owners, default → file → explicit CLI). - **Dummy client.** It has no `Client` and talks to the real client over RPC; it can adopt the same pattern later. - **Runtime settings the master sends over RPC** (`fsdir`, disk eviction, quota). They aren't startup inputs and stay where they are. - Hot reload, a public grouped config-file schema, and renaming flags, keys or environment variables. ### Process-wide state Several client settings are consumed by process-wide singletons or caches: - `ShmHelper`, `SpdkWrapper` and the registered pinned-memory manager; - the `call_once` mmap arena; - function-local statics for fileread / NoF worker pools, replica selection, object checksum, and RPC client I/O threads. Because of these, two `RealClient` instances in one process already cannot use different values for those settings. `ClientConfig` keeps them in their groups, but the follow-up distribution PRs mark them **process-scoped**. Those PRs initialize them once from the first validated config and log a warning if a later config differs, rather than silently ignoring it. Moving them to per-instance ownership is out of scope. ### Delivery plan Each step is its own PR, stacked on the previous one, with no behavior change beyond the validation timing listed above. 1. **Entry normalization and validation (client).** This step: - adds `ClientConfig` with `Validate()`; - adds the entry adapters and `RealClient::Setup(const ClientConfig&)`; - routes all five real-client entry points through it; - moves the early checks up front and adds tests for every precedence row above. Components are not changed yet. Because `Environ::Process()` snapshots the environment on first read (#4388), the values validated at the entry are the same values the components read later. 2. **Distribute: offload subtree.** `FileStorage` and `CreateStorageBackend` receive `offload` instead of reading Bucket / FilePerKey / OffsetAllocator / DFS / OSS / NVMe-KV environment settings. This also removes the per-call NVMe-KV reads. 3. **Distribute: transfer subtree.** `TransferSubmitter`, the worker pools and NoF. 4. **Distribute: memory and master-connection subtrees.** - Memory: hugepage and mmap arena, including the alloc/free re-reads. - Master connection: `MasterClient` pool config and replica selection. These are currently read inline in headers. - Remaining `RealClient` / `Client` reads. When step 4 is done, no component under the client reads external configuration directly. ### Relationship to existing work - **#3809:** environment convergence (Phase 1) and the catalog rule. This RFC builds on the typed, injectable environment reader in #4388 and the per-process catalog in #4392. - **#4360:** the same ownership principle on the master side. The two RFCs agree on: - no precedence or default changes; - one resolved value per setting; - business code receiving typed configs instead of `FLAGS_*` or environment lookups. ### Open questions 1. Should `ClientConfig` live under `mooncake-store/include/config` (public to pybind and the C API) or be split into a public input struct and a private resolved struct? 2. When a process-scoped setting differs between two clients in the same process, should that be a warning, as proposed, or a hard error? ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi added the

RFC

label

2026-10-29 21:22:57 +00:00

bsakaguchi commented

2026-10-29 21:22:57 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Sep 29, 2026):

Thanks for opening this issue, @Aionw!

Field	Value
Issue	#4396
GitHub user ID	`41376987`
Reporter	@Aionw

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Sep 29, 2026): Thanks for opening this issue, @Aionw! | Field | Value | |-------|-------| | **Issue** | #4396 | | **GitHub user ID** | `41376987` | | **Reporter** | @Aionw | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.