mirror of
https://github.com/deepseek-ai/3FS.git
synced 2026-11-05 00:47:47 +00:00
[GH-ISSUE #127] The three-node cluster failed to fully utilize the NIC bandwidth #73
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/3FS#73
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @RainSnowIceRain on GitHub (Mar 7, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/127
We deployed a three-node cluster, where one node serves as the client and metadata server, while the other two nodes function as storage nodes. We configured the system with a dual-replication strategy, set the chunk to 1MB, and the stripe to 16. We utilized the USRBIO FIO with the following parameters: -numjobs=128 -fallocate=none -iodepth=256 -iodepth_batch_submit=256 -iodepth_batch_complete_min=256 -iodepth_batch_complete_max=256 -direct=1 -rw=read -bs=1M --group_reporting -size=500M -mountpoint=/3fs/stage. However, neither the CPU, DRAM, NIC, nor the SSDs on the client/metadata node and storage nodes reached their maximum utilization. The 100Gbps NIC only achieved an actual throughput of approximately 40Gbps. Is this issue attributable to suboptimal cluster deployment methods, inadequate 3FS configurations, or improper fio settings?