[GH-ISSUE #1138] [Usage]: Does Mooncake Store support model metadata isolation? #377

New issue

Closed

opened 2026-10-29 21:13:35 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 21:13:35 +00:00

Owner

Originally created by @wushengtzy on GitHub (Nov 28, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1138

Describe your usage question

For example, can Qwen3-235B-A22B-INstruct and DeepSeeker V3.1 in vllm use the same mooncanke_master?

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @wushengtzy on GitHub (Nov 28, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1138 ### Describe your usage question For example, can Qwen3-235B-A22B-INstruct and DeepSeeker V3.1 in vllm use the same mooncanke_master? ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi closed this issue

2026-10-29 21:13:35 +00:00

bsakaguchi commented

2026-10-29 21:13:36 +00:00

Author

Owner

@tianlang-wq commented on GitHub (Nov 28, 2025):

This should have nothing to do with mooncake. You can understand it as simply being a Redis - what data is written into it depends on client behavior. In other words, whether model isolation is supported is a question for lmcache. Based on the analysis of lmcache's key format, it does support isolation between models. Below is the key format used by lmcache:

def to_string(self):
s = (
f"{self.fmt}@{self.model_name}@{self.world_size}"
f"@{self.worker_id}@{self.chunk_hash}"
)
if self.tags is not None and len(self.tags) != 0:
tags = [f"{k}%{v}" for k, v in self.tags.items()]
s += "@" + "@".join(tags)
return s

@tianlang-wq commented on GitHub (Nov 28, 2025): This should have nothing to do with mooncake. You can understand it as simply being a Redis - what data is written into it depends on client behavior. In other words, whether model isolation is supported is a question for lmcache. Based on the analysis of lmcache's key format, it does support isolation between models. Below is the key format used by lmcache: def to_string(self): s = ( f"{self.fmt}@{self.model_name}@{self.world_size}" f"@{self.worker_id}@{self.chunk_hash}" ) if self.tags is not None and len(self.tags) != 0: tags = [f"{k}%{v}" for k, v in self.tags.items()] s += "@" + "@".join(tags) return s

bsakaguchi commented

2026-10-29 21:13:36 +00:00

Author

Owner

@stmatengss commented on GitHub (Nov 29, 2025):

Actually, it supports multi-model (i.e., model metadata isolation) in integration with sglang/hicache. An example is here:

# NODE1
mooncake_master --enable_http_metadata_server=true --http_metadata_server_port=8080 --eviction_high_watermark_ratio=0.95 

# NODE2
MOONCAKE_TE_META_DATA_SERVER="http://NODE1/metadata" \
MOONCAKE_MASTER="NODE1:50051" \
python3 -m sglang.launch_server \
    --enable-hierarchical-cache \
    --hicache-storage-backend mooncake\
    --hicache-storage-backend-extra-config '{"storage_backend_tag": "model_name_1"}' \
    --model-path Qwen2.5-2B-Instruct

# NODE3
MOONCAKE_TE_META_DATA_SERVER="http://NODE1/metadata" \
MOONCAKE_MASTER="NODE1:50051" \
python3 -m sglang.launch_server \
    --enable-hierarchical-cache \
    --hicache-storage-backend mooncake\
    --hicache-storage-backend-extra-config '{"storage_backend_tag": "model_name_2"}' \
    --model-path Qwen2.5-2B-Instruct

@stmatengss commented on GitHub (Nov 29, 2025): Actually, it supports multi-model (i.e., model metadata isolation) in integration with sglang/hicache. An example is here: ``` # NODE1 mooncake_master --enable_http_metadata_server=true --http_metadata_server_port=8080 --eviction_high_watermark_ratio=0.95 # NODE2 MOONCAKE_TE_META_DATA_SERVER="http://NODE1/metadata" \ MOONCAKE_MASTER="NODE1:50051" \ python3 -m sglang.launch_server \ --enable-hierarchical-cache \ --hicache-storage-backend mooncake\ --hicache-storage-backend-extra-config '{"storage_backend_tag": "model_name_1"}' \ --model-path Qwen2.5-2B-Instruct # NODE3 MOONCAKE_TE_META_DATA_SERVER="http://NODE1/metadata" \ MOONCAKE_MASTER="NODE1:50051" \ python3 -m sglang.launch_server \ --enable-hierarchical-cache \ --hicache-storage-backend mooncake\ --hicache-storage-backend-extra-config '{"storage_backend_tag": "model_name_2"}' \ --model-path Qwen2.5-2B-Instruct ```

bsakaguchi commented

2026-10-29 21:13:36 +00:00

Author

Owner

@wushengtzy commented on GitHub (Dec 1, 2025):

Thanks for your reply

@wushengtzy commented on GitHub (Dec 1, 2025): Thanks for your reply

bsakaguchi commented

2026-10-29 21:13:36 +00:00

Author

Owner

@mjwtom commented on GitHub (Jul 15, 2026):

can not find storage_backend_tag in hicache

it should be extra_backend_tag

@mjwtom commented on GitHub (Jul 15, 2026): can not find storage_backend_tag in hicache it should be extra_backend_tag

bsakaguchi referenced this issue

2026-10-29 21:28:11 +00:00

[PR #377] [MERGED] chore: bump version to 0.3.1 in pyproject.toml #1526