[GH-ISSUE #3487] perf(store): batch io_uring reads in BucketStorageBackend #893

New issue

Closed

opened 2026-10-29 21:20:37 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:20:37 +00:00

Owner

Originally created by @982945902 on GitHub (Aug 17, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3487

Problem

BucketStorageBackend::BatchLoad() groups requested objects by bucket, but the io_uring path then calls read_aligned() once per key. Each call submits its SQEs and immediately waits for completion before the next key is processed.

This keeps cross-key queue depth at one even though UringFile::batch_read() already exists for submitting up to 32 independent reads together. The existing batch API only returns aggregate bytes, so it cannot safely identify per-key CQE failures or short reads.

Proposed behavior

Submit the aligned reads for one bucket through UringFile::batch_read().
Associate every CQE with its own descriptor, independent of completion order.
Drain all CQEs in the submitted batch before returning an error.
Report completed, bytes_read, and error per descriptor.
Reject a bucket load if any descriptor fails or does not cover the requested value range.
Preserve the synchronous BatchLoad() API; only device-side execution becomes batched.

SSD benchmark

Development-machine SSD, ext4, O_DIRECT, registered buffers, one thread, queue depth 32. Sequential and batch modes use the same file, buffers, random offsets, and alternate execution order.

Request size	Sequential	Batched	Speedup
4 KiB	14,433 IOPS / 56.38 MiB/s	198,277 IOPS / 774.52 MiB/s	13.74x
64 KiB	7,586 IOPS / 474.16 MiB/s	107,861 IOPS / 6,741.33 MiB/s	14.22x
1 MiB	3,377 IOPS / 3,376.94 MiB/s	18,695 IOPS / 18,694.75 MiB/s	5.54x

The benchmark is intended for relative comparison on the same machine; absolute throughput depends on the SSD/controller/filesystem.

Validation needed

More than 32 requests to cover multiple ring submissions.
Per-request short-read reporting.
CQE error draining followed by a successful operation on the same thread-local ring.
Bucket-level O_DIRECT integration with aligned destination buffers.
Build compatibility with and without liburing.

Originally created by @982945902 on GitHub (Aug 17, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3487 ## Problem `BucketStorageBackend::BatchLoad()` groups requested objects by bucket, but the io_uring path then calls `read_aligned()` once per key. Each call submits its SQEs and immediately waits for completion before the next key is processed. This keeps cross-key queue depth at one even though `UringFile::batch_read()` already exists for submitting up to 32 independent reads together. The existing batch API only returns aggregate bytes, so it cannot safely identify per-key CQE failures or short reads. ## Proposed behavior - Submit the aligned reads for one bucket through `UringFile::batch_read()`. - Associate every CQE with its own descriptor, independent of completion order. - Drain all CQEs in the submitted batch before returning an error. - Report `completed`, `bytes_read`, and `error` per descriptor. - Reject a bucket load if any descriptor fails or does not cover the requested value range. - Preserve the synchronous `BatchLoad()` API; only device-side execution becomes batched. ## SSD benchmark Development-machine SSD, ext4, O_DIRECT, registered buffers, one thread, queue depth 32. Sequential and batch modes use the same file, buffers, random offsets, and alternate execution order. | Request size | Sequential | Batched | Speedup | | --- | ---: | ---: | ---: | | 4 KiB | 14,433 IOPS / 56.38 MiB/s | 198,277 IOPS / 774.52 MiB/s | 13.74x | | 64 KiB | 7,586 IOPS / 474.16 MiB/s | 107,861 IOPS / 6,741.33 MiB/s | 14.22x | | 1 MiB | 3,377 IOPS / 3,376.94 MiB/s | 18,695 IOPS / 18,694.75 MiB/s | 5.54x | The benchmark is intended for relative comparison on the same machine; absolute throughput depends on the SSD/controller/filesystem. ## Validation needed - More than 32 requests to cover multiple ring submissions. - Per-request short-read reporting. - CQE error draining followed by a successful operation on the same thread-local ring. - Bucket-level O_DIRECT integration with aligned destination buffers. - Build compatibility with and without liburing.

bsakaguchi closed this issue

2026-10-29 21:20:37 +00:00

bsakaguchi commented

2026-10-29 21:20:38 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Aug 17, 2026):

Thanks for opening this issue, @982945902!

Field	Value
Issue	#3487
GitHub user ID	`49055103`
Reporter	@982945902

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Aug 17, 2026): Thanks for opening this issue, @982945902! | Field | Value | |-------|-------| | **Issue** | #3487 | | **GitHub user ID** | `49055103` | | **Reporter** | @982945902 | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi referenced this issue

2026-10-29 21:31:07 +00:00

[PR #893] [MERGED] [Docs] Update quick start and usage examples #1887