[GH-ISSUE #14] Questions about thread pool and shard #12

New issue

Closed

opened 2026-10-29 21:07:51 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 21:07:51 +00:00

Owner

Originally created by @CharlesJhonson on GitHub (Dec 3, 2024).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/14

Thanks for sharing the paper and code of this project! I have a question about the code:
What is the purpose of setting shard when there is already a worker_pool? What is the basis for setting the number to 8?

Originally created by @CharlesJhonson on GitHub (Dec 3, 2024). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/14 Thanks for sharing the paper and code of this project! I have a question about the code: What is the purpose of setting shard when there is already a worker_pool? What is the basis for setting the number to 8?

bsakaguchi closed this issue

2026-10-29 21:07:51 +00:00

bsakaguchi commented

2026-10-29 21:07:51 +00:00

Author

Owner

@alogfans commented on GitHub (Dec 4, 2024):

The shard mechanism is designed to evenly distribute the user's RDMA requests among the different threads of the worker pool. In addition, if context_.cqCount() is greater than one, the shard mechanism also mitigates concurrent contention for some RDMA resources (QP and doorbell), which has more significant performance impact in small I/Os.

Setting the number to 8 is based on our experience.

@alogfans commented on GitHub (Dec 4, 2024): The shard mechanism is designed to evenly distribute the user's RDMA requests among the different threads of the worker pool. In addition, if context_.cqCount() is greater than one, the shard mechanism also mitigates concurrent contention for some RDMA resources (QP and doorbell), which has more significant performance impact in small I/Os. Setting the number to 8 is based on our experience.

bsakaguchi commented

2026-10-29 21:07:52 +00:00

Author

Owner

@CharlesJhonson commented on GitHub (Dec 4, 2024):

Thank you for your answer.
If we want to avoid competition between QPs, shouldn't we consider assigning a CQ to each QP?

@CharlesJhonson commented on GitHub (Dec 4, 2024): Thank you for your answer. If we want to avoid competition between QPs, shouldn't we consider assigning a CQ to each QP?

bsakaguchi commented

2026-10-29 21:07:52 +00:00

Author

Owner

@alogfans commented on GitHub (Dec 5, 2024):

Yes, so Mooncake supports allocating multiple CQs per device, which are assigned to QPs in a round-robin manner.

@alogfans commented on GitHub (Dec 5, 2024): Yes, so Mooncake supports allocating multiple CQs per device, which are assigned to QPs in a round-robin manner.

bsakaguchi commented

2026-10-29 21:07:52 +00:00

Author

Owner

@CharlesJhonson commented on GitHub (Dec 5, 2024):

Ok , thank you very much!

@CharlesJhonson commented on GitHub (Dec 5, 2024): Ok , thank you very much!

bsakaguchi referenced this issue

2026-10-29 21:17:35 +00:00

[GH-ISSUE #2440] [Bug]: Execute mooncake+hixl, final destruction phase core #646

bsakaguchi referenced this issue

2026-10-29 21:26:03 +00:00