mirror of
https://github.com/kvcache-ai/Mooncake.git
synced 2026-11-05 01:17:55 +00:00
[GH-ISSUE #6] performance of the transfer_engine_bench tcp test is very poor #6
Labels
No labels
ascend
auto-closed
bug
bug
CI/Build
Common
documentation
enhancement
feature request
good first issue
help wanted
high priority
MetaX
nightly-failure
pull-request
PyTorch Backend
question
RFC
Roadmap
stale
Store
Tests
Transfer Engine
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/Mooncake#6
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @bohr on GitHub (Nov 28, 2024).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/6
when i use tow H20 machine test transfer_engine_bench, i not use rdma,only tcp, trans from cpu DRAM to cpu DRAM,but get poor performance,only 0.26GB/s
I1128 09:42:12.219687 229063 transfer_engine_bench.cpp:185] Worker 9 stopped!
I1128 09:42:12.220674 229057 transfer_engine_bench.cpp:185] Worker 3 stopped!
I1128 09:42:12.220672 229058 transfer_engine_bench.cpp:185] Worker 4 stopped!
I1128 09:42:12.220996 229059 transfer_engine_bench.cpp:185] Worker 5 stopped!
I1128 09:42:12.221199 229056 transfer_engine_bench.cpp:185] Worker 2 stopped!
I1128 09:42:12.221235 229062 transfer_engine_bench.cpp:185] Worker 8 stopped!
I1128 09:42:12.221266 229055 transfer_engine_bench.cpp:185] Worker 1 stopped!
I1128 09:42:12.221391 229054 transfer_engine_bench.cpp:185] Worker 0 stopped!
I1128 09:42:12.221467 229061 transfer_engine_bench.cpp:185] Worker 7 stopped!
I1128 09:42:12.221508 229060 transfer_engine_bench.cpp:185] Worker 6 stopped!
I1128 09:42:12.221588 229049 transfer_engine_bench.cpp:296] Test completed: duration 10.01, batch count 4895, throughput 0.26
@alogfans commented on GitHub (Nov 29, 2024):
The TCP mode is based on socket APIs, which is not for high-performance I/O. Each data transfer involves user/kernel switching, multiple buffer copying, etc., a throughput rate of only 200 MB/s is usually normal. The Transfer Engine performs best under RDMA.
@bohr commented on GitHub (Dec 2, 2024):
In this image, the TCP mode demonstrates an impressive throughput rate exceeding 30 GB/s. How can this experiment be reproduced?
@alogfans commented on GitHub (Dec 4, 2024):
The above figure is based on our environment, with four 200Gbps RoCE NICs installed per machine. It is reasonable that the throughput is only 200MB/s if you use on-board Ethernet NIC (roughly 2000Mbps). BTW, the TCP mode is used for demonstration rather than performance evaluation.