kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #644] [Usage]: MooncakeStore #209

New issue

Closed

opened 2026-10-29 21:10:55 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-29 21:10:55 +00:00

Owner

Originally created by @longyuetang on GitHub (Jul 19, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/644

Describe your usage question

In MooncakeStore, the hash value of the input tokens is truncated and used as the key. What if a hash collision occurs? Wouldn't that corrupt the KV cache data?

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @longyuetang on GitHub (Jul 19, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/644 ### Describe your usage question In MooncakeStore, the hash value of the input tokens is truncated and used as the key. What if a hash collision occurs? Wouldn't that corrupt the KV cache data? ### Before submitting a new issue... - [ ] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi closed this issue

2026-10-29 21:10:56 +00:00

bsakaguchi commented

2026-10-29 21:10:56 +00:00

Author

Owner

@ykwd commented on GitHub (Jul 20, 2025):

As far as I know, the prefix hash of the input tokens, which is used as the key for KV-cache indexing, is not truncated. Could you clarify where exactly the truncation is happening?

@ykwd commented on GitHub (Jul 20, 2025): As far as I know, the prefix hash of the input tokens, which is used as the key for KV-cache indexing, is not truncated. Could you clarify where exactly the truncation is happening?

bsakaguchi commented

2026-10-29 21:10:56 +00:00

Author

Owner

@longyuetang commented on GitHub (Jul 20, 2025):

As far as I know, the prefix hash of the input tokens, which is used as the key for KV-cache indexing, is not truncated. Could you clarify where exactly the truncation is happening?

@longyuetang commented on GitHub (Jul 20, 2025): > As far as I know, the prefix hash of the input tokens, which is used as the key for KV-cache indexing, is not truncated. Could you clarify where exactly the truncation is happening? <img width="780" height="573" alt="Image" src="https://github.com/user-attachments/assets/ea9b978b-f9f3-4414-99d4-51521c3d68cf" /> <img width="731" height="156" alt="Image" src="https://github.com/user-attachments/assets/e10a885a-6c0b-4c9c-9362-3f6435480e8d" />

bsakaguchi commented

2026-10-29 21:10:56 +00:00

Author

Owner

@ykwd commented on GitHub (Jul 21, 2025):

@ShangmingCai Could you take a look? Thanks.

@ykwd commented on GitHub (Jul 21, 2025): @ShangmingCai Could you take a look? Thanks.

bsakaguchi commented

2026-10-29 21:10:56 +00:00

Author

Owner

@ShangmingCai commented on GitHub (Jul 21, 2025):

@jialei-jialei Yeah, this is possible, all hash method has this issue, but the chance is very tiny. Just like the block hash in vllm page attention. BTW, if current_tokens of different requests are identical, it means their prompts are identical, that's when we reuse kvcache. You'd better check what input_tokens_tensor really means.

@ShangmingCai commented on GitHub (Jul 21, 2025): @jialei-jialei Yeah, this is possible, all hash method has this issue, but the chance is very tiny. Just like the block hash in vllm page attention. BTW, if `current_tokens` of different requests are identical, it means their prompts are identical, that's when we reuse kvcache. You'd better check what `input_tokens_tensor` really means.

bsakaguchi commented

2026-10-29 21:10:57 +00:00

Author

Owner

@longyuetang commented on GitHub (Jul 23, 2025):

@jialei-jialei Yeah, this is possible, all hash method has this issue, but the chance is very tiny. Just like the block hash in vllm page attention. BTW, if current_tokens of different requests are identical, it means their prompts are identical, that's when we reuse kvcache. You'd better check what input_tokens_tensor really means.

My point is, if different current_tokens end up with identical hash values, MoonStore would retrieve the wrong data. This is what I'm questioning. I’d like to understand—even if the probability is extremely low, would such a case still be negligible in practice?

@longyuetang commented on GitHub (Jul 23, 2025): > [@jialei-jialei](https://github.com/jialei-jialei) Yeah, this is possible, all hash method has this issue, but the chance is very tiny. Just like the block hash in vllm page attention. BTW, if `current_tokens` of different requests are identical, it means their prompts are identical, that's when we reuse kvcache. You'd better check what `input_tokens_tensor` really means. My point is, if different current_tokens end up with identical hash values, MoonStore would retrieve the wrong data. This is what I'm questioning. I’d like to understand—even if the probability is extremely low, would such a case still be negligible in practice?

bsakaguchi commented

2026-10-29 21:10:57 +00:00

Author

Owner

@ShangmingCai commented on GitHub (Jul 23, 2025):

@jialei-jialei Yes, I think it is negligible. Although we didn't split kvcache at the block level for this version, you can refer to the content-hash-based impl of https://github.com/vllm-project/vllm/blob/main/vllm/core/block/prefix_caching_block.py#L54-L67, which shares a similar hash-based design.

@ShangmingCai commented on GitHub (Jul 23, 2025): @jialei-jialei Yes, I think it is negligible. Although we didn't split kvcache at the block level for this version, you can refer to the content-hash-based impl of https://github.com/vllm-project/vllm/blob/main/vllm/core/block/prefix_caching_block.py#L54-L67, which shares a similar hash-based design.

bsakaguchi referenced this issue

2026-10-29 21:27:05 +00:00

[PR #209] [MERGED] [CI] Update GitHub Actions permissions for build-and-release job #1396