[GH-ISSUE #1157] [Performance]: Measured throughput exceeds max theoretical bandwidth #388

New issue

Closed

opened 2026-10-29 21:13:45 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:13:45 +00:00

Owner

Originally created by @hzt123123 on GitHub (Dec 3, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1157

Describe your performance question

When running ./transfer_engine_bench on two H20 servers, the throughput is 213 GB/s, but the max theoretical bandwidth is 200GB/s. Additionally, throughput is highly unstable, varying between 190 GB/s and 213 GB/s.

The experimental details are as follows:
export MC_NUM_QP_PER_EP=1

./transfer_engine_bench --metadata_server=etcd://xxx --mode=initiator --segment_id=abcd:12345 --operation=read --device_name=mlx5_bond_1,mlx5_bond_2,mlx5_bond_3,mlx5_bond_4,mlx5_bond_5,mlx5_bond_6,mlx5_bond_7,mlx5_bond_8 --threads=16 --batch_size=128 --block_size=131072

./transfer_engine_bench --mode=target --metadata_server=etcd://xxx --local_server_name=abcd:12345 --operation=read --device_name=mlx5_bond_1,mlx5_bond_2,mlx5_bond_3,mlx5_bond_4,mlx5_bond_5,mlx5_bond_6,mlx5_bond_7,mlx5_bond_8 --threads=16 --block_size=131072

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @hzt123123 on GitHub (Dec 3, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1157 ### Describe your performance question When running ./transfer_engine_bench on two H20 servers, the throughput is 213 GB/s, but the max theoretical bandwidth is 200GB/s. Additionally, throughput is highly unstable, varying between 190 GB/s and 213 GB/s. The experimental details are as follows: export MC_NUM_QP_PER_EP=1 ./transfer_engine_bench --metadata_server=etcd://xxx --mode=initiator --segment_id=abcd:12345 --operation=read --device_name=mlx5_bond_1,mlx5_bond_2,mlx5_bond_3,mlx5_bond_4,mlx5_bond_5,mlx5_bond_6,mlx5_bond_7,mlx5_bond_8 --threads=16 --batch_size=128 --block_size=131072 ./transfer_engine_bench --mode=target --metadata_server=etcd://xxx --local_server_name=abcd:12345 --operation=read --device_name=mlx5_bond_1,mlx5_bond_2,mlx5_bond_3,mlx5_bond_4,mlx5_bond_5,mlx5_bond_6,mlx5_bond_7,mlx5_bond_8 --threads=16 --block_size=131072 ### Before submitting a new issue... - [ ] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi closed this issue

2026-10-29 21:13:45 +00:00

bsakaguchi commented

2026-10-29 21:13:46 +00:00

Author

Owner

@alogfans commented on GitHub (Dec 4, 2025):

Closed as it is duplicated as #1158

@alogfans commented on GitHub (Dec 4, 2025): Closed as it is duplicated as #1158

bsakaguchi referenced this issue

2026-10-29 21:28:14 +00:00

[PR #388] [MERGED] make backlog size of handshake listen configurable #1532