[GH-ISSUE #1303] [Bug]: Significant performance degradation in Transfer Engine #429

New issue

Closed

opened 2026-10-29 21:14:23 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:14:23 +00:00

Owner

Originally created by @chestnut-Q on GitHub (Dec 29, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1303

Originally assigned to: @staryxchen on GitHub.

Bug Report

The transfer engine on the main branch has a significant throughput degradation (about 50%). The following are the test results from transfer_engine_bench.cpp:

main (27GB/s)

commit 38138f9136 (47GB/s)

Through bisection, the issue was traced to changes in !1130, which introduced a global lock in the frequently-polled getTransferStatus method, causing this performance degradation.

cc @staryxchen @alogfans

Before submitting...

Ensure you searched for relevant issues and read the [documentation]

Originally created by @chestnut-Q on GitHub (Dec 29, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1303 Originally assigned to: @staryxchen on GitHub. ### Bug Report The transfer engine on the main branch has a significant throughput degradation (about 50%). The following are the test results from transfer_engine_bench.cpp: * main (27GB/s) <img width="1107" height="101" alt="Image" src="https://github.com/user-attachments/assets/5a9a5bea-5d9c-4c30-9c85-d0af040250bb" /> * commit 38138f913645a3e8abcf0dac89d742e21dad3473 (47GB/s) <img width="1099" height="74" alt="Image" src="https://github.com/user-attachments/assets/be39cd1c-bbaf-4163-a3f5-db84dfbb6464" /> Through bisection, the issue was traced to changes in [!1130](https://github.com/kvcache-ai/Mooncake/pull/1130), which introduced a global lock in the frequently-polled getTransferStatus method, causing this performance degradation. cc @staryxchen @alogfans ### Before submitting... - [x] Ensure you searched for relevant issues and read the [documentation]

bsakaguchi

2026-10-29 21:14:23 +00:00

closed this issue
added the
high priority

bug
labels

bsakaguchi commented

2026-10-29 21:14:24 +00:00

Author

Owner

@staryxchen commented on GitHub (Dec 29, 2025):

Thank you for reporting this issue🙏. I will resolve it as soon as possible.

@staryxchen commented on GitHub (Dec 29, 2025): Thank you for reporting this issue🙏. I will resolve it as soon as possible.