[GH-ISSUE #352] Does 3FS fuse client support multiple InfiniBand NIC? 3FS的fuse客户端支持多网卡吗? #215

Closed
opened 2026-10-26 09:47:15 +00:00 by bsakaguchi · 1 comment
Owner

Originally created by @csyanxing on GitHub (Oct 17, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/352

We deploy a 3fs cluster in our machines, each machine has 2 400Gbps InfiniBand NICs, which are all connected to an IB switch. However, when we use fio to test read speed in one client machine (the client machine is also equipped with 2 400Gbps NICs), the read speed is about 40GiB/s, which seems to be close to the bandwith of one single IB NIC. So we wonder why the bandwidth is not fully used, and does 3fs fuse client support multiple IB NIC?

我们尝试部署了一套3fs文件系统,每台机器有2张400Gbps的InfiniBand网卡,通过IB交换机相连,部署完成后在一台单独的客户端机器(客户端机器同样是2张400Gbps网卡)用fio进行文件读取测试,所测得的最大读取速率是大约40GiB/s,看起来只有一张网卡的带宽被打满,两张网卡的带宽并没有完全被充分利用,因此我们想知道3fs的客户端是否支持多网卡同时连接和文件传输?

Originally created by @csyanxing on GitHub (Oct 17, 2025). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/352 We deploy a 3fs cluster in our machines, each machine has 2 400Gbps InfiniBand NICs, which are all connected to an IB switch. However, when we use fio to test read speed in one client machine (the client machine is also equipped with 2 400Gbps NICs), the read speed is about 40GiB/s, which seems to be close to the bandwith of one single IB NIC. So we wonder why the bandwidth is not fully used, and does 3fs fuse client support multiple IB NIC? 我们尝试部署了一套3fs文件系统,每台机器有2张400Gbps的InfiniBand网卡,通过IB交换机相连,部署完成后在一台单独的客户端机器(客户端机器同样是2张400Gbps网卡)用fio进行文件读取测试,所测得的最大读取速率是大约40GiB/s,看起来只有一张网卡的带宽被打满,两张网卡的带宽并没有完全被充分利用,因此我们想知道3fs的客户端是否支持多网卡同时连接和文件传输?
Author
Owner

@463975089 commented on GitHub (Dec 26, 2025):

请问ib卡的toml文件如何配置,

<!-- gh-comment-id:3692448677 --> @463975089 commented on GitHub (Dec 26, 2025): 请问ib卡的toml文件如何配置,
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/3FS#215
No description provided.