deepseek-ai/3FS
1
0
Fork 0
mirror of https://github.com/deepseek-ai/3FS.git synced 2026-11-05 00:47:47 +00:00
Code Issues 166 Projects Packages Wiki Activity

[GH-ISSUE #98] Unable to Deploy 3FS Without RDMA – Consider Virtualizing an RDMA Network #55

New issue
Open
opened 2026-10-26 09:44:16 +00:00 by bsakaguchi · 11 comments
bsakaguchi commented 2026-10-26 09:44:16 +00:00
Owner
Copy link

Originally created by @nfyang-nexus on GitHub (Mar 5, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/98

In our standard network deployment environment, it is not possible to deploy 3FS directly because RDMA networking is mandatory. Therefore, we can consider virtualizing an RDMA network environment to meet the requirements.

Steps to Set Up a Virtual RDMA Environment

​ 1. Install Required Packages

sudo apt update  
sudo apt install -y rdma-core infiniband-diags ibverbs-utils

​ 2. Load the rxe Kernel Module

modprobe rdma_rxe

​ 3. Verify That the Module Is Loaded

lsmod | grep rxe

Expected output:

rdma_rxe              139264  0  
ip6_udp_tunnel         16384  1 rdma_rxe  
udp_tunnel             20480  1 rdma_rxe  
ib_uverbs             163840  2 rdma_rxe,rdma_ucm  
ib_core               393216  6 rdma_cm,rdma_rxe,iw_cm,rdma_ucm,ib_uverbs,ib_cm

​ 4. Check Available Network Interfaces

ip link show

Example output:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000  
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00  
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000  
    link/ether 52:54:00:71:4b:d8 brd ff:ff:ff:ff:ff:ff  
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default  
    link/ether 3e:25:8e:73:c6:9a brd ff:ff:ff:ff:ff:ff

​ • lo: Loopback interface for local communication (127.0.0.1), not a physical NIC.

​ • enp1s0: The primary network interface (Ethernet), status UP, MAC address 52:54:00:71:4b:d8.

​ • docker0: Docker virtual bridge, currently DOWN, meaning no active container connections.

​ 5. Add enp1s0 as an RDMA Device

rdma link add rxe0 type rxe netdev enp1s0

​ 6. Verify RDMA Device Creation

rdma link show

If successful, you should see the new rxe0 device, which serves as the virtual RDMA version of enp1s0.

Run 3FS with the Virtualized RDMA Network

Once the virtual RDMA environment is set up, you can proceed with starting 3FS:

./bin/monitor_collector_main -cfg ./etc/monitor_collector_main.toml

This should now work as expected in an environment where physical RDMA hardware is unavailable.

Originally created by @nfyang-nexus on GitHub (Mar 5, 2025). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/98 In our standard network deployment environment, it is not possible to deploy **3FS** directly because **RDMA networking is mandatory**. Therefore, we can consider **virtualizing an RDMA network environment** to meet the requirements. **Steps to Set Up a Virtual RDMA Environment** ​ 1. **Install Required Packages** ``` sudo apt update sudo apt install -y rdma-core infiniband-diags ibverbs-utils ``` ​ 2. **Load the rxe Kernel Module** ``` modprobe rdma_rxe ``` ​ 3. **Verify That the Module Is Loaded** ``` lsmod | grep rxe ``` Expected output: ``` rdma_rxe 139264 0 ip6_udp_tunnel 16384 1 rdma_rxe udp_tunnel 20480 1 rdma_rxe ib_uverbs 163840 2 rdma_rxe,rdma_ucm ib_core 393216 6 rdma_cm,rdma_rxe,iw_cm,rdma_ucm,ib_uverbs,ib_cm ``` ​ 4. **Check Available Network Interfaces** ``` ip link show ``` Example output: ``` 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether 52:54:00:71:4b:d8 brd ff:ff:ff:ff:ff:ff 3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default link/ether 3e:25:8e:73:c6:9a brd ff:ff:ff:ff:ff:ff ``` ​ • **lo**: Loopback interface for local communication (127.0.0.1), not a physical NIC. ​ • **enp1s0**: The primary network interface (Ethernet), status **UP**, MAC address 52:54:00:71:4b:d8. ​ • **docker0**: Docker virtual bridge, currently **DOWN**, meaning no active container connections. ​ 5. **Add enp1s0 as an RDMA Device** ``` rdma link add rxe0 type rxe netdev enp1s0 ``` ​ 6. **Verify RDMA Device Creation** ``` rdma link show ``` If successful, you should see the new **rxe0** device, which serves as the virtual RDMA version of **enp1s0**. **Run 3FS with the Virtualized RDMA Network** Once the virtual RDMA environment is set up, you can proceed with starting **3FS**: ``` ./bin/monitor_collector_main -cfg ./etc/monitor_collector_main.toml ``` This should now work as expected in an environment where physical RDMA hardware is unavailable.
bsakaguchi commented 2026-10-26 09:44:17 +00:00
Author
Owner
Copy link

@SSmallMonster commented on GitHub (Mar 5, 2025):

so great, this makes me 'can touch 3fs'!

<!-- gh-comment-id:2700353865 --> @SSmallMonster commented on GitHub (Mar 5, 2025): so great, this makes me 'can touch 3fs'!
bsakaguchi commented 2026-10-26 09:44:17 +00:00
Author
Owner
Copy link

@SSmallMonster commented on GitHub (Mar 5, 2025):

If anyone occurs err like: modprobe: ERROR: could not insert ‘rdma_rxe’: Invalid argument, try to uninstall mlnx_ofed package and reboot(to reload modules).

Some symbols build version in rdma_rxe is conflict with the version in the kernel module ib_core which imported from mlnx package.

<!-- gh-comment-id:2700871033 --> @SSmallMonster commented on GitHub (Mar 5, 2025): If anyone occurs err like: `modprobe: ERROR: could not insert ‘rdma_rxe’: Invalid argument`, try to uninstall mlnx_ofed package and reboot(to reload modules). Some symbols build version in `rdma_rxe` is conflict with the version in the kernel module `ib_core` which imported from mlnx package.
bsakaguchi commented 2026-10-26 09:44:17 +00:00
Author
Owner
Copy link

@zartbot commented on GitHub (Mar 5, 2025):

AlibabaCloud provide all region and all az support for RDMA on 8th generation generic computing instance with high performance. Here is a chinese deployment guide https://mp.weixin.qq.com/s/Ri9-sjsaoQ3LUE2-dTAN_g , just FYI.

<!-- gh-comment-id:2701086121 --> @zartbot commented on GitHub (Mar 5, 2025): AlibabaCloud provide all region and all az support for RDMA on 8th generation generic computing instance with high performance. Here is a chinese deployment guide https://mp.weixin.qq.com/s/Ri9-sjsaoQ3LUE2-dTAN_g , just FYI.
bsakaguchi commented 2026-10-26 09:44:17 +00:00
Author
Owner
Copy link

@xiaoyaoyouyue commented on GitHub (Mar 6, 2025):

I install the rdma-core, and reboot, but modeprobe FATAL , do you know the reason ? thanks!

root@ubuntu:# modprobe rdma_rxe
modprobe: FATAL: Module rdma_rxe not found in directory /lib/modules/5.15.0-1073-kvm
root@ubuntu:# sudo apt install -y rdma-core infiniband-diags ibverbs-utils
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
rdma-core is already the newest version (39.0-1).
ibverbs-utils is already the newest version (39.0-1).
infiniband-diags is already the newest version (39.0-1).
0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded.
root@ubuntu:~# modprobe rdma_rxe
modprobe: FATAL: Module rdma_rxe not found in directory /lib/modules/5.15.0-1073-kvm

<!-- gh-comment-id:2702909502 --> @xiaoyaoyouyue commented on GitHub (Mar 6, 2025): I install the rdma-core, and reboot, but modeprobe FATAL , do you know the reason ? thanks! root@ubuntu:~# modprobe rdma_rxe modprobe: FATAL: Module rdma_rxe not found in directory /lib/modules/5.15.0-1073-kvm root@ubuntu:~# sudo apt install -y rdma-core infiniband-diags ibverbs-utils Reading package lists... Done Building dependency tree... Done Reading state information... Done rdma-core is already the newest version (39.0-1). ibverbs-utils is already the newest version (39.0-1). infiniband-diags is already the newest version (39.0-1). 0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded. root@ubuntu:~# modprobe rdma_rxe modprobe: FATAL: Module rdma_rxe not found in directory /lib/modules/5.15.0-1073-kvm
bsakaguchi commented 2026-10-26 09:44:17 +00:00
Author
Owner
Copy link

@SSmallMonster commented on GitHub (Mar 6, 2025):

I install the rdma-core, and reboot, but modeprobe FATAL , do you know the reason ? thanks!

root@ubuntu:# modprobe rdma_rxe modprobe: FATAL: Module rdma_rxe not found in directory /lib/modules/5.15.0-1073-kvm root@ubuntu:# sudo apt install -y rdma-core infiniband-diags ibverbs-utils Reading package lists... Done Building dependency tree... Done Reading state information... Done rdma-core is already the newest version (39.0-1). ibverbs-utils is already the newest version (39.0-1). infiniband-diags is already the newest version (39.0-1). 0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded. root@ubuntu:~# modprobe rdma_rxe modprobe: FATAL: Module rdma_rxe not found in directory /lib/modules/5.15.0-1073-kvm

try to install by apt install linux-modules-extra-$(uname -r), the module is provided by the linux-modules-extra packages.

<!-- gh-comment-id:2702934637 --> @SSmallMonster commented on GitHub (Mar 6, 2025): > I install the rdma-core, and reboot, but modeprobe FATAL , do you know the reason ? thanks! > > root@ubuntu:~# modprobe rdma_rxe modprobe: FATAL: Module rdma_rxe not found in directory /lib/modules/5.15.0-1073-kvm root@ubuntu:~# sudo apt install -y rdma-core infiniband-diags ibverbs-utils Reading package lists... Done Building dependency tree... Done Reading state information... Done rdma-core is already the newest version (39.0-1). ibverbs-utils is already the newest version (39.0-1). infiniband-diags is already the newest version (39.0-1). 0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded. root@ubuntu:~# modprobe rdma_rxe modprobe: FATAL: Module rdma_rxe not found in directory /lib/modules/5.15.0-1073-kvm try to install by `apt install linux-modules-extra-$(uname -r)`, the module is provided by the **linux-modules-extra packages**.
bsakaguchi commented 2026-10-26 09:44:18 +00:00
Author
Owner
Copy link

@freedomsunny commented on GitHub (Mar 7, 2025):

great!

<!-- gh-comment-id:2705958407 --> @freedomsunny commented on GitHub (Mar 7, 2025): great!
bsakaguchi commented 2026-10-26 09:44:18 +00:00
Author
Owner
Copy link

@xiaoyaoyouyue commented on GitHub (Mar 8, 2025):

(base) root@ubuntu:/3fs# apt install linux-modules-extra-$(uname -r)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package linux-modules-extra-5.15.0-1073-kvm
E: Couldn't find any package by glob 'linux-modules-extra-5.15.0-1073-kvm'
(base) root@ubuntu:/3fs#

命令尝试如上，另外我尝试了使用升级内核的方式，结果系统升级挂了

<!-- gh-comment-id:2708143688 --> @xiaoyaoyouyue commented on GitHub (Mar 8, 2025): (base) root@ubuntu:~/3fs# apt install linux-modules-extra-$(uname -r) Reading package lists... Done Building dependency tree... Done Reading state information... Done E: Unable to locate package linux-modules-extra-5.15.0-1073-kvm E: Couldn't find any package by glob 'linux-modules-extra-5.15.0-1073-kvm' (base) root@ubuntu:~/3fs# 命令尝试如上，另外我尝试了使用升级内核的方式，结果系统升级挂了
bsakaguchi commented 2026-10-26 09:44:18 +00:00
Author
Owner
Copy link

@xiaoyaoyouyue commented on GitHub (Mar 11, 2025):

(base) root@ubuntu:/3fs# apt install linux-modules-extra-$(uname -r) Reading package lists... Done Building dependency tree... Done Reading state information... Done E: Unable to locate package linux-modules-extra-5.15.0-1073-kvm E: Couldn't find any package by glob 'linux-modules-extra-5.15.0-1073-kvm' (base) root@ubuntu:/3fs#

命令尝试如上，另外我尝试了使用升级内核的方式，结果系统升级挂了

使用generic的系统可以了

<!-- gh-comment-id:2712747587 --> @xiaoyaoyouyue commented on GitHub (Mar 11, 2025): > (base) root@ubuntu:~/3fs# apt install linux-modules-extra-$(uname -r) Reading package lists... Done Building dependency tree... Done Reading state information... Done E: Unable to locate package linux-modules-extra-5.15.0-1073-kvm E: Couldn't find any package by glob 'linux-modules-extra-5.15.0-1073-kvm' (base) root@ubuntu:~/3fs# > > 命令尝试如上，另外我尝试了使用升级内核的方式，结果系统升级挂了 使用generic的系统可以了
bsakaguchi commented 2026-10-26 09:44:19 +00:00
Author
Owner
Copy link

@Icedroid commented on GitHub (Mar 13, 2025):

I run in docker , but mgmt not start successfully. How cloud I do to resolve it?

IBDevice.cc:386 WARNING] IfAddr of mlx5_0:1 -> Bond0 not found, maybe running in container!
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.437726968+00:00 mgmtd_main:   32 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.437769976+00:00 mgmtd_main:   32 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE >
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.437778035+00:00 mgmtd_main:   32 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.442579863+00:00 mgmtd_main:   32 IBDevice.cc:386 WARNING] IfAddr of mlx5_1:1 -> Bond0 not found, maybe running in container!
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.442583943+00:00 mgmtd_main:   32 IBDevice.cc:441 CRITICAL] IBDevice mlx5_1:1 can't set zone by IP, fallback to UNKNOWN
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.442604000+00:00 mgmtd_main:   32 IBDevice.cc:367 INFO] IBDevice mlx5_1 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE >
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.442606911+00:00 mgmtd_main:   32 IBDevice.cc:256 INFO] IBDevice add mlx5_1, id 1, 1 available ports
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.448109240+00:00 mgmtd_main:   32 IBDevice.cc:386 WARNING] IfAddr of mlx5_2:1 -> Bond0 not found, maybe running in container!
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.448113464+00:00 mgmtd_main:   32 IBDevice.cc:441 CRITICAL] IBDevice mlx5_2:1 can't set zone by IP, fallback to UNKNOWN
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.448134390+00:00 mgmtd_main:   32 IBDevice.cc:367 INFO] IBDevice mlx5_2 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE >
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.448136644+00:00 mgmtd_main:   32 IBDevice.cc:256 INFO] IBDevice add mlx5_2, id 2, 1 available ports
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.452865290+00:00 mgmtd_main:   32 IBDevice.cc:386 WARNING] IfAddr of mlx5_3:1 -> Bond0 not found, maybe running in container!
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.452869181+00:00 mgmtd_main:   32 IBDevice.cc:441 CRITICAL] IBDevice mlx5_3:1 can't set zone by IP, fallback to UNKNOWN
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.452888769+00:00 mgmtd_main:   32 IBDevice.cc:367 INFO] IBDevice mlx5_3 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE >
Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.452891236+00:00 mgmtd_main:   32 IBDevice.cc:256 INFO] IBDevice add mlx5_3, id 3, 1 available ports
<!-- gh-comment-id:2721105721 --> @Icedroid commented on GitHub (Mar 13, 2025): I run in docker , but mgmt not start successfully. How cloud I do to resolve it? ``` IBDevice.cc:386 WARNING] IfAddr of mlx5_0:1 -> Bond0 not found, maybe running in container! Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.437726968+00:00 mgmtd_main: 32 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.437769976+00:00 mgmtd_main: 32 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE > Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.437778035+00:00 mgmtd_main: 32 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.442579863+00:00 mgmtd_main: 32 IBDevice.cc:386 WARNING] IfAddr of mlx5_1:1 -> Bond0 not found, maybe running in container! Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.442583943+00:00 mgmtd_main: 32 IBDevice.cc:441 CRITICAL] IBDevice mlx5_1:1 can't set zone by IP, fallback to UNKNOWN Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.442604000+00:00 mgmtd_main: 32 IBDevice.cc:367 INFO] IBDevice mlx5_1 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE > Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.442606911+00:00 mgmtd_main: 32 IBDevice.cc:256 INFO] IBDevice add mlx5_1, id 1, 1 available ports Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.448109240+00:00 mgmtd_main: 32 IBDevice.cc:386 WARNING] IfAddr of mlx5_2:1 -> Bond0 not found, maybe running in container! Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.448113464+00:00 mgmtd_main: 32 IBDevice.cc:441 CRITICAL] IBDevice mlx5_2:1 can't set zone by IP, fallback to UNKNOWN Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.448134390+00:00 mgmtd_main: 32 IBDevice.cc:367 INFO] IBDevice mlx5_2 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE > Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.448136644+00:00 mgmtd_main: 32 IBDevice.cc:256 INFO] IBDevice add mlx5_2, id 2, 1 available ports Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.452865290+00:00 mgmtd_main: 32 IBDevice.cc:386 WARNING] IfAddr of mlx5_3:1 -> Bond0 not found, maybe running in container! Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.452869181+00:00 mgmtd_main: 32 IBDevice.cc:441 CRITICAL] IBDevice mlx5_3:1 can't set zone by IP, fallback to UNKNOWN Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.452888769+00:00 mgmtd_main: 32 IBDevice.cc:367 INFO] IBDevice mlx5_3 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE > Mar 13 11:10:09 localhost.localdomain mgmtd_main[32]: [2025-03-13T11:10:09.452891236+00:00 mgmtd_main: 32 IBDevice.cc:256 INFO] IBDevice add mlx5_3, id 3, 1 available ports ```
bsakaguchi commented 2026-10-26 09:44:19 +00:00
Author
Owner
Copy link

@ghost commented on GitHub (Mar 23, 2025):

ibdev2netdev This file is required

$ which ibdev2netdev
/usr/sbin/ibdev2netdev

root@vm:~# sudo  ibv_devinfo
hca_id:	rxe0
	transport:			InfiniBand (0)
	fw_ver:				0.0.0
	node_guid:			0a00:27ff:fe67:4f90
	sys_image_guid:			0a00:27ff:fe67:4f90
	vendor_id:			0xffffff
	vendor_part_id:			0
	hw_ver:				0x0
	phys_port_cnt:			1
		port:	1
			state:			PORT_ACTIVE (4)
			max_mtu:		4096 (5)
			active_mtu:		1024 (3)
			sm_lid:			0
			port_lid:		0
			port_lmc:		0x00
			link_layer:		Ethernet

$ sudo  rdma link list
link rxe0/1 state ACTIVE physical_state LINK_UP netdev enp0s8
<!-- gh-comment-id:2746182419 --> @ghost commented on GitHub (Mar 23, 2025): ### ibdev2netdev This file is required * https://network.nvidia.com/products/infiniband-drivers/linux/mlnx_ofed/ * install ofed dirver ```sh $ which ibdev2netdev /usr/sbin/ibdev2netdev ``` ```sh root@vm:~# sudo ibv_devinfo hca_id: rxe0 transport: InfiniBand (0) fw_ver: 0.0.0 node_guid: 0a00:27ff:fe67:4f90 sys_image_guid: 0a00:27ff:fe67:4f90 vendor_id: 0xffffff vendor_part_id: 0 hw_ver: 0x0 phys_port_cnt: 1 port: 1 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 1024 (3) sm_lid: 0 port_lid: 0 port_lmc: 0x00 link_layer: Ethernet $ sudo rdma link list link rxe0/1 state ACTIVE physical_state LINK_UP netdev enp0s8 ```
bsakaguchi commented 2026-10-26 09:44:19 +00:00
Author
Owner
Copy link

@SimonCqk commented on GitHub (Apr 25, 2025):

Is there anyone works on native TCP network environment?

<!-- gh-comment-id:2829314141 --> @SimonCqk commented on GitHub (Apr 25, 2025): Is there anyone works on native TCP network environment?
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
SF-Zhou-patch-1
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/3FS#55
No description provided.