[GH-ISSUE #2980] [Bug]: File-per-key path mapping can collide and return another object's data #769

New issue

Open

opened 2026-10-29 21:19:19 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:19:19 +00:00

Owner

Originally created by @feichai0017 on GitHub (Jul 17, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2980

Bug Report

On current main (f20b7061), ResolvePathFromKey() is not collision-safe for file-per-key storage.

The path uses only the low 8 bits of std::hash<std::string> for its two directory levels, while SanitizeKey() maps each of /\\:*?"<>| to _. For a fixed tenant, the 10,000 four-character keys built from those characters plus _ all sanitize to the same filename but have only 256 possible directory pairs, so at least two distinct keys must map to the same path.

This is a correctness problem because:

writes open the destination with O_TRUNC, so the later key overwrites the earlier file;
StorageBackendAdaptor::BatchLoad() deserializes the stored KVEntry without checking that its key matches the requested key;
when the colliding keys and values have equal lengths, a read can silently return the other key's value.

The same path helper is used by file-per-key LOCAL_DISK storage and the legacy DISK replica path.

Expected behavior

Distinct object identities must never resolve to the same on-disk object path in normal operation, and reads must reject records whose stored key does not match the requested key.

Proposed direction

Use a stable full-key digest for the filename, retain a legacy-path read fallback for existing files, and validate the deserialized key before copying data to the caller.

Before submitting...

Searched existing issues and open PRs for path/key collision reports.

Originally created by @feichai0017 on GitHub (Jul 17, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2980 ## Bug Report On current `main` (`f20b7061`), `ResolvePathFromKey()` is not collision-safe for file-per-key storage. The path uses only the low 8 bits of `std::hash<std::string>` for its two directory levels, while `SanitizeKey()` maps each of `/\\:*?"<>|` to `_`. For a fixed tenant, the 10,000 four-character keys built from those characters plus `_` all sanitize to the same filename but have only 256 possible directory pairs, so at least two distinct keys must map to the same path. This is a correctness problem because: - writes open the destination with `O_TRUNC`, so the later key overwrites the earlier file; - `StorageBackendAdaptor::BatchLoad()` deserializes the stored `KVEntry` without checking that its key matches the requested key; - when the colliding keys and values have equal lengths, a read can silently return the other key's value. The same path helper is used by file-per-key LOCAL_DISK storage and the legacy DISK replica path. ## Expected behavior Distinct object identities must never resolve to the same on-disk object path in normal operation, and reads must reject records whose stored key does not match the requested key. ## Proposed direction Use a stable full-key digest for the filename, retain a legacy-path read fallback for existing files, and validate the deserialized key before copying data to the caller. ### Before submitting... - [x] Searched existing issues and open PRs for path/key collision reports.

bsakaguchi commented

2026-10-29 21:19:20 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 17, 2026):

Thanks for opening this issue, @feichai0017!

Field	Value
Issue	#2980
GitHub user ID	`132118805`
Reporter	@feichai0017

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Jul 17, 2026): Thanks for opening this issue, @feichai0017! | Field | Value | |-------|-------| | **Issue** | #2980 | | **GitHub user ID** | `132118805` | | **Reporter** | @feichai0017 | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi referenced this issue

2026-10-29 21:30:35 +00:00

[PR #779] [MERGED] [TE] Fix notifs problems in C wrapper #1816