[GH-ISSUE #1033] [Performance]: High CPU usage due to busy-wait in TransferEngineOperationState::wait_for_completion #337

New issue

Closed

opened 2026-10-29 21:12:57 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:12:57 +00:00

Owner

Originally created by @wwq2333 on GitHub (Nov 7, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1033

Originally assigned to: @Copilot on GitHub.

Describe your performance question

Describe

TransferEngineOperationState::wait_for_completion() performs busy-waiting: it loops indefinitely and repeatedly calls check_task_status() without any wait/yield/backoff. Under high RDMA latency or bandwidth saturation, this can peg a CPU core and hurt overall throughput when CPU is constrained.

Impact

Sustained high CPU usage by the waiting thread(s) during large/long transfers.
Lower system throughput under CPU contention;

Real-world scenario: Offline inference, throughput-first

Network: Large data streaming easily saturates the RDMA NIC; under saturation or transient congestion, transfer completion latency increases, so the current tight polling keeps spinning for long periods.

CPU contention: Each waiting thread can peg a CPU core. This competes with CPU-heavy preprocessing stages (tokenization, chunking/sharding, mmap reads, decoding).

Resource efficiency: Spinning wastes CPU cycles that could be used for data preparation, further affecting subsequent GPU inference and leading to throughput degradation.

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @wwq2333 on GitHub (Nov 7, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1033 Originally assigned to: @Copilot on GitHub. ### Describe your performance question ### Describe TransferEngineOperationState::[wait_for_completion](https://github.com/kvcache-ai/Mooncake/blob/6d05c9344ba4522144ca4b22106cc3fc8661fdb1/mooncake-store/src/transfer_task.cpp#L291)() performs busy-waiting: it loops indefinitely and repeatedly calls [check_task_status()](https://github.com/kvcache-ai/Mooncake/blob/6d05c9344ba4522144ca4b22106cc3fc8661fdb1/mooncake-store/src/transfer_task.cpp#L224) without any wait/yield/backoff. Under high RDMA latency or bandwidth saturation, this can peg a CPU core and hurt overall throughput when CPU is constrained. ### Impact - Sustained high CPU usage by the waiting thread(s) during large/long transfers. - Lower system throughput under CPU contention; ### Real-world scenario: Offline inference, throughput-first Network: Large data streaming easily saturates the RDMA NIC; under saturation or transient congestion, transfer completion latency increases, so the current tight polling keeps spinning for long periods. CPU contention: Each waiting thread can peg a CPU core. This competes with CPU-heavy preprocessing stages (tokenization, chunking/sharding, mmap reads, decoding). Resource efficiency: Spinning wastes CPU cycles that could be used for data preparation, further affecting subsequent GPU inference and leading to throughput degradation. ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi closed this issue

2026-10-29 21:12:57 +00:00

bsakaguchi referenced this issue

2026-10-29 21:27:55 +00:00

[PR #337] [MERGED] fix(store): the metadata leak after umount segment #1490