kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #1071] [Bug]: Receiver side race condition #357

New issue

Closed

opened 2026-10-29 21:13:17 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-29 21:13:17 +00:00

Owner

Originally created by @ShangmingCai on GitHub (Nov 18, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1071

Originally assigned to: @alogfans on GitHub.

Bug Report

context: https://github.com/sgl-project/sglang/pull/13231

cc: @alogfans

Before submitting...

Ensure you searched for relevant issues and read the [documentation]

Originally created by @ShangmingCai on GitHub (Nov 18, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1071 Originally assigned to: @alogfans on GitHub. ### Bug Report context: https://github.com/sgl-project/sglang/pull/13231 cc: @alogfans ### Before submitting... - [x] Ensure you searched for relevant issues and read the [documentation]

bsakaguchi

2026-10-29 21:13:17 +00:00

closed this issue
added the
stale

auto-closed

bug
labels

bsakaguchi commented

2026-10-29 21:13:17 +00:00

Author

Owner

@ShangmingCai commented on GitHub (Nov 18, 2025):

@flying-x Thank you so much for the digging. I think we can discuss and arrange a fix here.

@ShangmingCai commented on GitHub (Nov 18, 2025): @flying-x Thank you so much for the digging. I think we can discuss and arrange a fix here.

bsakaguchi commented

2026-10-29 21:13:18 +00:00

Author

Owner

@alogfans commented on GitHub (Nov 18, 2025):

@flying-x So the fix scope should be inside tcp_transport, especially the ASIO callback logic. Try to check that if there are multiple slices for a single request.

@alogfans commented on GitHub (Nov 18, 2025): @flying-x So the fix scope should be inside tcp_transport, especially the ASIO callback logic. Try to check that if there are multiple slices for a single request.

bsakaguchi commented

2026-10-29 21:13:18 +00:00

Author

Owner

@flying-x commented on GitHub (Nov 18, 2025):

Thank you @ShangmingCai and @alogfans.

I am thinking that the sender side can queue a readBody call after sending all slices and when the callback of the readBody is done, it then calls markSuccess(). On the receiving side, once all readBody calls are done (including cudaMemcpy calls), it will send an ACK msg (1 byte, with writeBody and then finish the request) so that readBody call from the sender side will be unblocked.

The sender side needs to have a timeout on the readBody, but other than that, there should be no new error handling.

Does the above sound OK?

@flying-x commented on GitHub (Nov 18, 2025): Thank you @ShangmingCai and @alogfans. I am thinking that the sender side can queue a readBody call after sending all slices and when the callback of the readBody is done, it then calls markSuccess(). On the receiving side, once all readBody calls are done (including cudaMemcpy calls), it will send an ACK msg (1 byte, with writeBody and then finish the request) so that readBody call from the sender side will be unblocked. The sender side needs to have a timeout on the readBody, but other than that, there should be no new error handling. Does the above sound OK?

bsakaguchi commented

2026-10-29 21:13:18 +00:00

Author

Owner

@ShangmingCai commented on GitHub (Nov 19, 2025):

This will definitely increase the TTFT, but I assume TCP fallback doesn't require strict SLO for TTFT.

As long as we can keep most changes in the mooncake part, this method seems reasonable to me.

@ShangmingCai commented on GitHub (Nov 19, 2025): This will definitely increase the TTFT, but I assume TCP fallback doesn't require strict SLO for TTFT. As long as we can keep most changes in the mooncake part, this method seems reasonable to me.

bsakaguchi commented

2026-10-29 21:13:18 +00:00

Author

Owner

@ykwd commented on GitHub (Nov 21, 2025):

In fact, I previously encountered a similar issue when testing with TCP. After the TE reports that the transfer has completed successfully, if you immediately check the data on the receiving side, there is a chance that the data has not been fully written yet.

@ykwd commented on GitHub (Nov 21, 2025): In fact, I previously encountered a similar issue when testing with TCP. After the TE reports that the transfer has completed successfully, if you immediately check the data on the receiving side, there is a chance that the data has not been fully written yet.

bsakaguchi commented

2026-10-29 21:13:18 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 19, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jun 19, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:13:18 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 26, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jun 26, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

bsakaguchi referenced this issue

2026-10-29 21:28:03 +00:00

[PR #357] [MERGED] fix wrong command in XpYd #1508