deepseek-ai/3FS
1
0
Fork 0
mirror of https://github.com/deepseek-ai/3FS.git synced 2026-11-05 00:47:47 +00:00
Code Issues 166 Projects Packages Wiki Activity

[GH-ISSUE #78] [Build Guide] build 3FS with docker #43

New issue
Open
opened 2026-10-26 09:44:00 +00:00 by bsakaguchi · 21 comments
bsakaguchi commented 2026-10-26 09:44:00 +00:00
Owner
Copy link

Originally created by @Wine93 on GitHub (Mar 4, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/78

Thanks to deepseek, we have access to the outstanding filesystem 3FS.
If you want to compile or use 3FS, here is a guide to compiling with docker for you.

Step 1: create container

run command in host

$ mkdir build-3fs; cd build-3fs
$ wget https://gist.githubusercontent.com/Wine93/8faa3e2545873fb58f70b058ccf0987f/raw/6d92a64ca6ff7e9a954bbe74e01b593678881cee/playground.sh
$ sudo bash playground.sh

Step2: install dependencies

run command in container

2.1 install dependencies

$ apt update
$ apt install -y cmake libuv1-dev liblz4-dev liblzma-dev libdouble-conversion-dev libdwarf-dev libunwind-dev \
    libaio-dev libgflags-dev libgoogle-glog-dev libgtest-dev libgmock-dev clang-format-14 clang-14 clang-tidy-14 lld-14 \
    libgoogle-perftools-dev google-perftools libssl-dev gcc-12 g++-12 libboost-all-dev cargo git g++ wget meson

2.2 install foundationdb

$ cd ${BUILD_DIR}
$ wget https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-server_7.1.67-1_amd64.deb \
    https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-clients_7.1.67-1_amd64.deb 
$ dpkg -i foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb

2.3 build and install fuse

$ cd ${BUILD_DIR}
$ wget https://github.com/libfuse/libfuse/releases/download/fuse-3.16.2/fuse-3.16.2.tar.gz
$ tar -zxvf fuse-3.16.2.tar.gz
$ cd fuse-3.16.2; mkdir build; cd build
$ meson setup ..
$ ninja
$ ninja install  # ignore error

Step 3: build 3FS

run command in container

$ cd ${BUILD_DIR}
$ git clone https://github.com/deepseek-ai/3fs
$ cd 3fs
$ git submodule update --init --recursive
$ ./patches/apply.sh
$ cmake -S . -B build -DCMAKE_CXX_COMPILER=clang++-14 -DCMAKE_C_COMPILER=clang-14 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
$ cmake --build build -j 45

It will generates binrary shown as below once build success:

$ cd ${BUILD_DIR}; ls -ls 3fs/build/bin
total 2308428
355344 -rwxr-xr-x 1 root root 363871904 Mar  4 11:36 admin_cli
144976 -rwxr-xr-x 1 root root 148454880 Mar  4 11:30 hf3fs-admin
204336 -rwxr-xr-x 1 root root 209239320 Mar  4 11:32 hf3fs_fuse_main
277812 -rwxr-xr-x 1 root root 284476352 Mar  4 11:30 meta_main
174700 -rwxr-xr-x 1 root root 178892200 Mar  4 11:27 mgmtd_main
168300 -rwxr-xr-x 1 root root 172336688 Mar  4 11:26 migration_main
102740 -rwxr-xr-x 1 root root 105205000 Mar  4 11:19 monitor_collector_main
170628 -rwxr-xr-x 1 root root 174721688 Mar  4 11:26 simple_example_main
395964 -rwxr-xr-x 1 root root 405484072 Mar  4 11:34 storage_bench
313628 -rwxr-xr-x 1 root root 321173936 Mar  4 11:28 storage_main
Originally created by @Wine93 on GitHub (Mar 4, 2025). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/78 Thanks to deepseek, we have access to the outstanding filesystem 3FS. If you want to compile or use 3FS, here is a guide to compiling with docker for you. # Step 1: create container > run command in host ```bash $ mkdir build-3fs; cd build-3fs $ wget https://gist.githubusercontent.com/Wine93/8faa3e2545873fb58f70b058ccf0987f/raw/6d92a64ca6ff7e9a954bbe74e01b593678881cee/playground.sh $ sudo bash playground.sh ``` # Step2: install dependencies > run command in container ## 2.1 install dependencies ```bash $ apt update $ apt install -y cmake libuv1-dev liblz4-dev liblzma-dev libdouble-conversion-dev libdwarf-dev libunwind-dev \ libaio-dev libgflags-dev libgoogle-glog-dev libgtest-dev libgmock-dev clang-format-14 clang-14 clang-tidy-14 lld-14 \ libgoogle-perftools-dev google-perftools libssl-dev gcc-12 g++-12 libboost-all-dev cargo git g++ wget meson ``` ## 2.2 install foundationdb ```shell $ cd ${BUILD_DIR} $ wget https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-server_7.1.67-1_amd64.deb \ https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-clients_7.1.67-1_amd64.deb $ dpkg -i foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb ``` ## 2.3 build and install fuse ```bash $ cd ${BUILD_DIR} $ wget https://github.com/libfuse/libfuse/releases/download/fuse-3.16.2/fuse-3.16.2.tar.gz $ tar -zxvf fuse-3.16.2.tar.gz $ cd fuse-3.16.2; mkdir build; cd build $ meson setup .. $ ninja $ ninja install # ignore error ``` # Step 3: build 3FS > run command in container ```bash $ cd ${BUILD_DIR} $ git clone https://github.com/deepseek-ai/3fs $ cd 3fs $ git submodule update --init --recursive $ ./patches/apply.sh $ cmake -S . -B build -DCMAKE_CXX_COMPILER=clang++-14 -DCMAKE_C_COMPILER=clang-14 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON $ cmake --build build -j 45 ``` It will generates binrary shown as below once build success: ```shell $ cd ${BUILD_DIR}; ls -ls 3fs/build/bin total 2308428 355344 -rwxr-xr-x 1 root root 363871904 Mar 4 11:36 admin_cli 144976 -rwxr-xr-x 1 root root 148454880 Mar 4 11:30 hf3fs-admin 204336 -rwxr-xr-x 1 root root 209239320 Mar 4 11:32 hf3fs_fuse_main 277812 -rwxr-xr-x 1 root root 284476352 Mar 4 11:30 meta_main 174700 -rwxr-xr-x 1 root root 178892200 Mar 4 11:27 mgmtd_main 168300 -rwxr-xr-x 1 root root 172336688 Mar 4 11:26 migration_main 102740 -rwxr-xr-x 1 root root 105205000 Mar 4 11:19 monitor_collector_main 170628 -rwxr-xr-x 1 root root 174721688 Mar 4 11:26 simple_example_main 395964 -rwxr-xr-x 1 root root 405484072 Mar 4 11:34 storage_bench 313628 -rwxr-xr-x 1 root root 321173936 Mar 4 11:28 storage_main ```
bsakaguchi commented 2026-10-26 09:44:01 +00:00
Author
Owner
Copy link

@shuiyihang commented on GitHub (Mar 4, 2025):

Thanks for your guide, I used the compiled program in the container as follows:

/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml

This error has occurred. Have you encountered it

/root/.profile: line 10: /.cargo/env: No such file or directory
[2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up)
[2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6
[2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true
[2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true
[2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24
[2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt
[2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports
[2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started.
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true,
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ]
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}}
[2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
*** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') ***
*** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: ***
[2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
@ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv
/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449
@ 000000000004251f (unknown)
@ 00000000000969fc pthread_kill
@ 0000000000042475 raise
@ 00000000000287f2 abort
@ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE
/3fs/third_party/folly/folly/logging/LogCategory.cpp:71
@ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190
@ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222
@ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token
/3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49
@ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74
-> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc
@ 00000000000dc252 (unknown)
@ 0000000000094ac2 (unknown)
@ 0000000000125a03 clone
Aborted (core dumped)

<!-- gh-comment-id:2696390893 --> @shuiyihang commented on GitHub (Mar 4, 2025): Thanks for your guide, I used the compiled program in the container as follows: `/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml` This error has occurred. Have you encountered it > /root/.profile: line 10: /.cargo/env: No such file or directory [2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up) [2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6 [2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true [2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true [2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24 [2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN [2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt [2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports [2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started. [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true, [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [ [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal" [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ] [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL", [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr" [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } [2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}} [2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) *** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') *** *** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: *** [2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) [2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) [2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) [2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) [2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) @ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv /3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449 @ 000000000004251f (unknown) @ 00000000000969fc pthread_kill @ 0000000000042475 raise @ 00000000000287f2 abort @ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE /3fs/third_party/folly/folly/logging/LogCategory.cpp:71 @ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190 @ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222 @ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49 @ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74 -> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc @ 00000000000dc252 (unknown) @ 0000000000094ac2 (unknown) @ 0000000000125a03 clone Aborted (core dumped)
bsakaguchi commented 2026-10-26 09:44:02 +00:00
Author
Owner
Copy link

@baijing0818 commented on GitHub (Mar 4, 2025):

根据帖子提示构建的Dockerfile，你们可以尝试构建：

# 使用 Ubuntu 22.04 作为基础镜像
FROM ubuntu:22.04

# 设置非交互模式，避免 apt 交互提示
ENV DEBIAN_FRONTEND=noninteractive

# 更新并安装 3FS 构建所需的依赖
RUN apt update && apt install -y \
    cmake \
    libuv1-dev \
    liblz4-dev \
    liblzma-dev \
    libdouble-conversion-dev \
    libdwarf-dev \
    libunwind-dev \
    libaio-dev \
    libgflags-dev \
    libgoogle-glog-dev \
    libgtest-dev \
    libgmock-dev \
    clang-format-14 \
    clang-14 \
    clang-tidy-14 \
    lld-14 \
    libgoogle-perftools-dev \
    google-perftools \
    libssl-dev \
    gcc-12 \
    g++-12 \
    libboost-all-dev \
    cargo \
    git \
    wget \
    meson \
    ninja-build \
    dpkg \
    && rm -rf /var/lib/apt/lists/*

# 安装 FoundationDB
WORKDIR /tmp
RUN wget https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-server_7.1.67-1_amd64.deb \
    https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-clients_7.1.67-1_amd64.deb && \
    dpkg -i foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb && \
    rm foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb

# 构建并安装 fuse（忽略 ninja install 错误）
WORKDIR /tmp
RUN wget https://github.com/libfuse/libfuse/releases/download/fuse-3.16.2/fuse-3.16.2.tar.gz && \
    tar -zxvf fuse-3.16.2.tar.gz && \
    mkdir -p fuse-3.16.2/build && \
    cd fuse-3.16.2/build && \
    meson setup .. && \
    ninja && \
    ninja install || true

# 设置构建目录环境变量
ENV BUILD_DIR=/build

# 创建工作目录，并克隆 3FS 源码进行构建
WORKDIR ${BUILD_DIR}
RUN git clone https://github.com/deepseek-ai/3fs.git && \
    cd 3fs && \
    git submodule update --init --recursive && \
    ./patches/apply.sh && \
    cmake -S . -B build \
          -DCMAKE_CXX_COMPILER=clang++-14 \
          -DCMAKE_C_COMPILER=clang-14 \
          -DCMAKE_BUILD_TYPE=RelWithDebInfo \
          -DCMAKE_EXPORT_COMPILE_COMMANDS=ON && \
    cmake --build build -j $(nproc)

# 将 3FS 编译生成的 bin 目录加入 PATH
ENV PATH="${BUILD_DIR}/3fs/build/bin:${PATH}"

CMD ["/bin/bash"]

但是，我的思路是确保其能够类似 Ceph 进行多节点集群化部署，并直接管理本地磁盘，如果能借鉴 Ceph 的容器化架构，将 3FS 的各组件解耦成独立容器，但是采用同一镜像，不同的启动命令参数以启动不同角色容器，支持多节点集群部署，并利用 systemd 管理容器的生命周期，这样方便迁移和组件改造升级。

3FS（Fire-Flyer File System）由 集群管理（Cluster Manager） 、 元数据服务（Metadata Service） 、 存储服务（Storage Service） 和 客户端（Client） 四部分组成 (3FS/docs/design_notes.md at main · deepseek-ai/3FS · GitHub)。

那么比如：为存储服务编写/etc/systemd/system/3fs-storage@.service模板，
例如：

[Unit]
Description=3FS Cluster Manager (mgmtd) Container
After=docker.service
Requires=docker.service

[Service]
Restart=always
ExecStartPre=-/usr/bin/docker rm -f 3fs-mgmtd
ExecStart=/usr/bin/docker run --name 3fs-mgmtd --net=host --privileged \
    -v /etc/3fs:/opt/3fs/etc:ro \
    deepseek/3fs:latest mgmtd_main
ExecStop=/usr/bin/docker stop 3fs-mgmtd

[Install]
WantedBy=multi-user.target

使每块磁盘对应一个实例名启动（实例名可采用磁盘标识或节点ID）

docker run --name 3fs-storage-%i \
    --hostname %H \ 
    --net=host --ipc=host --ulimit nofile=1048576:1048576 \
    --privileged \
    -v /dev:/dev \ 
    -v /etc/3fs:/opt/3fs/etc:ro \
    -e NODE_ID=%i -e CLUSTER_ID=stage \
    deepseek/3fs:latest storage_main

然后mgmtd 组建容器类似：

docker run --name 3fs-mgmtd --net=host --privileged \
    -v /etc/3fs:/opt/3fs/etc:ro \
    deepseek/3fs:latest mgmtd_main

我感觉完全docker化目前还尚早，等市场反应，稍后应该会有各家集成。

<!-- gh-comment-id:2696579330 --> @baijing0818 commented on GitHub (Mar 4, 2025): 根据帖子提示构建的Dockerfile，你们可以尝试构建： ```bash # 使用 Ubuntu 22.04 作为基础镜像 FROM ubuntu:22.04 # 设置非交互模式，避免 apt 交互提示 ENV DEBIAN_FRONTEND=noninteractive # 更新并安装 3FS 构建所需的依赖 RUN apt update && apt install -y \ cmake \ libuv1-dev \ liblz4-dev \ liblzma-dev \ libdouble-conversion-dev \ libdwarf-dev \ libunwind-dev \ libaio-dev \ libgflags-dev \ libgoogle-glog-dev \ libgtest-dev \ libgmock-dev \ clang-format-14 \ clang-14 \ clang-tidy-14 \ lld-14 \ libgoogle-perftools-dev \ google-perftools \ libssl-dev \ gcc-12 \ g++-12 \ libboost-all-dev \ cargo \ git \ wget \ meson \ ninja-build \ dpkg \ && rm -rf /var/lib/apt/lists/* # 安装 FoundationDB WORKDIR /tmp RUN wget https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-server_7.1.67-1_amd64.deb \ https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-clients_7.1.67-1_amd64.deb && \ dpkg -i foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb && \ rm foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb # 构建并安装 fuse（忽略 ninja install 错误） WORKDIR /tmp RUN wget https://github.com/libfuse/libfuse/releases/download/fuse-3.16.2/fuse-3.16.2.tar.gz && \ tar -zxvf fuse-3.16.2.tar.gz && \ mkdir -p fuse-3.16.2/build && \ cd fuse-3.16.2/build && \ meson setup .. && \ ninja && \ ninja install || true # 设置构建目录环境变量 ENV BUILD_DIR=/build # 创建工作目录，并克隆 3FS 源码进行构建 WORKDIR ${BUILD_DIR} RUN git clone https://github.com/deepseek-ai/3fs.git && \ cd 3fs && \ git submodule update --init --recursive && \ ./patches/apply.sh && \ cmake -S . -B build \ -DCMAKE_CXX_COMPILER=clang++-14 \ -DCMAKE_C_COMPILER=clang-14 \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DCMAKE_EXPORT_COMPILE_COMMANDS=ON && \ cmake --build build -j $(nproc) # 将 3FS 编译生成的 bin 目录加入 PATH ENV PATH="${BUILD_DIR}/3fs/build/bin:${PATH}" CMD ["/bin/bash"] ``` 但是，我的思路是确保其能够类似 Ceph 进行多节点集群化部署，并直接管理本地磁盘，如果能借鉴 Ceph 的容器化架构，将 3FS 的各组件解耦成独立容器，但是采用同一镜像，不同的启动命令参数以启动不同角色容器，支持多节点集群部署，并利用 systemd 管理容器的生命周期，这样方便迁移和组件改造升级。 3FS（Fire-Flyer File System）由 **集群管理（Cluster Manager）** 、 **元数据服务（Metadata Service）** 、 **存储服务（Storage Service）** 和 **客户端（Client）** 四部分组成 ([3FS/docs/design_notes.md at main · deepseek-ai/3FS · GitHub](https://github.com/deepseek-ai/3FS/blob/main/docs/design_notes.md#:~:text=The%203FS%20system%20has%20four,InfiniBand%20or%20RoCE))。 那么比如：为存储服务编写/etc/systemd/system/3fs-storage@.service模板， 例如： ```bash [Unit] Description=3FS Cluster Manager (mgmtd) Container After=docker.service Requires=docker.service [Service] Restart=always ExecStartPre=-/usr/bin/docker rm -f 3fs-mgmtd ExecStart=/usr/bin/docker run --name 3fs-mgmtd --net=host --privileged \ -v /etc/3fs:/opt/3fs/etc:ro \ deepseek/3fs:latest mgmtd_main ExecStop=/usr/bin/docker stop 3fs-mgmtd [Install] WantedBy=multi-user.target ``` 使每块磁盘对应一个实例名启动（实例名可采用磁盘标识或节点ID） ```bash docker run --name 3fs-storage-%i \ --hostname %H \ --net=host --ipc=host --ulimit nofile=1048576:1048576 \ --privileged \ -v /dev:/dev \ -v /etc/3fs:/opt/3fs/etc:ro \ -e NODE_ID=%i -e CLUSTER_ID=stage \ deepseek/3fs:latest storage_main ``` 然后mgmtd 组建容器类似： ```bash docker run --name 3fs-mgmtd --net=host --privileged \ -v /etc/3fs:/opt/3fs/etc:ro \ deepseek/3fs:latest mgmtd_main ``` 我感觉完全docker化目前还尚早，等市场反应，稍后应该会有各家集成。
bsakaguchi commented 2026-10-26 09:44:02 +00:00
Author
Owner
Copy link

@baijing0818 commented on GitHub (Mar 4, 2025):

另外，追加一个精简镜像构建：

#####################################
# Stage 1：构建阶段 – 编译 3FS 二进制
#####################################
FROM ubuntu:22.04 AS builder

# 设置非交互模式
ENV DEBIAN_FRONTEND=noninteractive

# 安装编译所需依赖
RUN apt update && apt install -y \
    cmake \
    libuv1-dev \
    liblz4-dev \
    liblzma-dev \
    libdouble-conversion-dev \
    libdwarf-dev \
    libunwind-dev \
    libaio-dev \
    libgflags-dev \
    libgoogle-glog-dev \
    libgtest-dev \
    libgmock-dev \
    clang-format-14 \
    clang-14 \
    clang-tidy-14 \
    lld-14 \
    libgoogle-perftools-dev \
    google-perftools \
    libssl-dev \
    gcc-12 \
    g++-12 \
    libboost-all-dev \
    cargo \
    git \
    wget \
    meson \
    ninja-build \
    dpkg \
    && rm -rf /var/lib/apt/lists/*

# 安装 FoundationDB（用于元数据存储）
WORKDIR /tmp
RUN wget https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-server_7.1.67-1_amd64.deb \
    https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-clients_7.1.67-1_amd64.deb && \
    dpkg -i foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb && \
    rm foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb

# 构建 fuse（可选，如果你需要在客户端容器中使用 FUSE）
WORKDIR /tmp
RUN wget https://github.com/libfuse/libfuse/releases/download/fuse-3.16.2/fuse-3.16.2.tar.gz && \
    tar -zxvf fuse-3.16.2.tar.gz && \
    mkdir -p fuse-3.16.2/build && \
    cd fuse-3.16.2/build && \
    meson setup .. && \
    ninja && \
    ninja install || true

# 克隆并构建 3FS
ENV BUILD_DIR=/build
WORKDIR ${BUILD_DIR}
RUN git clone --depth 1 --recurse-submodules https://github.com/deepseek-ai/3fs.git && \
    cd 3fs && \
    ./patches/apply.sh && \
    cmake -S . -B build \
          -DCMAKE_CXX_COMPILER=clang++-14 \
          -DCMAKE_C_COMPILER=clang-14 \
          -DCMAKE_BUILD_TYPE=RelWithDebInfo \
          -DCMAKE_EXPORT_COMPILE_COMMANDS=ON && \
    cmake --build build -j $(nproc) && \
    # 剥除二进制调试信息以减小体积
    strip build/bin/*

#####################################
# Stage 2：运行时阶段 – 精简镜像
#####################################
FROM ubuntu:22.04

ENV DEBIAN_FRONTEND=noninteractive

# 安装运行时所需的最低依赖（仅安装必须的运行库）
RUN apt update && apt install -y \
    libfuse3-3 \
    libssl-dev \
    libuv1 \
    liblz4-1 \
    liblzma5 \
    libdouble-conversion3 \
    libdwarf1 \
    libunwind8 \
    libaio1 \
    libgflags2.2 \
    libgoogle-glog0v5 \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /opt/3fs

# 将构建阶段生成的 3FS 二进制复制到运行镜像中
COPY --from=builder /build/3fs/build/bin/ ./bin/
# 同时复制一份默认配置和文档（根据项目实际情况调整）
COPY --from=builder /build/3fs/configs/ ./etc/
COPY --from=builder /build/3fs/deploy/README.md ./README.md

# 设置 PATH，保证容器启动时可以直接调用 3FS 各服务二进制
ENV PATH="/opt/3fs/bin:${PATH}"

# 默认进入 bash，可在启动时通过指定命令来选择具体服务（如 mgmtd_main、meta_main、storage_main 等）
CMD ["/bin/bash"]

示例:

docker run --name 3fs-mgmtd --net=host --privileged 3fs-build-lite mgmtd_main

以及磁盘：

docker run --name 3fs-storage-10001 --net=host --privileged -v /dev:/dev 3fs-build-lite storage_main
<!-- gh-comment-id:2696660523 --> @baijing0818 commented on GitHub (Mar 4, 2025): 另外，追加一个精简镜像构建： ```bash ##################################### # Stage 1：构建阶段 – 编译 3FS 二进制 ##################################### FROM ubuntu:22.04 AS builder # 设置非交互模式 ENV DEBIAN_FRONTEND=noninteractive # 安装编译所需依赖 RUN apt update && apt install -y \ cmake \ libuv1-dev \ liblz4-dev \ liblzma-dev \ libdouble-conversion-dev \ libdwarf-dev \ libunwind-dev \ libaio-dev \ libgflags-dev \ libgoogle-glog-dev \ libgtest-dev \ libgmock-dev \ clang-format-14 \ clang-14 \ clang-tidy-14 \ lld-14 \ libgoogle-perftools-dev \ google-perftools \ libssl-dev \ gcc-12 \ g++-12 \ libboost-all-dev \ cargo \ git \ wget \ meson \ ninja-build \ dpkg \ && rm -rf /var/lib/apt/lists/* # 安装 FoundationDB（用于元数据存储） WORKDIR /tmp RUN wget https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-server_7.1.67-1_amd64.deb \ https://github.com/apple/foundationdb/releases/download/7.1.67/foundationdb-clients_7.1.67-1_amd64.deb && \ dpkg -i foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb && \ rm foundationdb-server_7.1.67-1_amd64.deb foundationdb-clients_7.1.67-1_amd64.deb # 构建 fuse（可选，如果你需要在客户端容器中使用 FUSE） WORKDIR /tmp RUN wget https://github.com/libfuse/libfuse/releases/download/fuse-3.16.2/fuse-3.16.2.tar.gz && \ tar -zxvf fuse-3.16.2.tar.gz && \ mkdir -p fuse-3.16.2/build && \ cd fuse-3.16.2/build && \ meson setup .. && \ ninja && \ ninja install || true # 克隆并构建 3FS ENV BUILD_DIR=/build WORKDIR ${BUILD_DIR} RUN git clone --depth 1 --recurse-submodules https://github.com/deepseek-ai/3fs.git && \ cd 3fs && \ ./patches/apply.sh && \ cmake -S . -B build \ -DCMAKE_CXX_COMPILER=clang++-14 \ -DCMAKE_C_COMPILER=clang-14 \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DCMAKE_EXPORT_COMPILE_COMMANDS=ON && \ cmake --build build -j $(nproc) && \ # 剥除二进制调试信息以减小体积 strip build/bin/* ##################################### # Stage 2：运行时阶段 – 精简镜像 ##################################### FROM ubuntu:22.04 ENV DEBIAN_FRONTEND=noninteractive # 安装运行时所需的最低依赖（仅安装必须的运行库） RUN apt update && apt install -y \ libfuse3-3 \ libssl-dev \ libuv1 \ liblz4-1 \ liblzma5 \ libdouble-conversion3 \ libdwarf1 \ libunwind8 \ libaio1 \ libgflags2.2 \ libgoogle-glog0v5 \ && rm -rf /var/lib/apt/lists/* WORKDIR /opt/3fs # 将构建阶段生成的 3FS 二进制复制到运行镜像中 COPY --from=builder /build/3fs/build/bin/ ./bin/ # 同时复制一份默认配置和文档（根据项目实际情况调整） COPY --from=builder /build/3fs/configs/ ./etc/ COPY --from=builder /build/3fs/deploy/README.md ./README.md # 设置 PATH，保证容器启动时可以直接调用 3FS 各服务二进制 ENV PATH="/opt/3fs/bin:${PATH}" # 默认进入 bash，可在启动时通过指定命令来选择具体服务（如 mgmtd_main、meta_main、storage_main 等） CMD ["/bin/bash"] ``` 示例: ```bash docker run --name 3fs-mgmtd --net=host --privileged 3fs-build-lite mgmtd_main ``` 以及磁盘： ```bash docker run --name 3fs-storage-10001 --net=host --privileged -v /dev:/dev 3fs-build-lite storage_main ```
bsakaguchi commented 2026-10-26 09:44:02 +00:00
Author
Owner
Copy link

@baijing0818 commented on GitHub (Mar 4, 2025):

感谢您的指导，我在容器中使用了编译好的程序，如下所示：

/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml

发生了此错误。你遇到过吗

/root/.profile: line 10: /.cargo/env: No such file or directory
[2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up)
[2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6
[2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true
[2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true
[2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24
[2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt
[2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports
[2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started.
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true,
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ]
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}}
[2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
*** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') ***
*** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: ***
[2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
@ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv
/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449
@ 000000000004251f (unknown)
@ 00000000000969fc pthread_kill
@ 0000000000042475 raise
@ 00000000000287f2 abort
@ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE
/3fs/third_party/folly/folly/logging/LogCategory.cpp:71
@ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190
@ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222
@ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token
/3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49
@ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74
-> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc
@ 00000000000dc252 (unknown)
@ 0000000000094ac2 (unknown)
@ 0000000000125a03 clone
Aborted (core dumped)

你网络硬件不支持，没有IB网卡吧，

docker run --privileged --device=/dev/infiniband:/dev/infiniband -it 3fs-build

注意设备

<!-- gh-comment-id:2696704685 --> @baijing0818 commented on GitHub (Mar 4, 2025): > 感谢您的指导，我在容器中使用了编译好的程序，如下所示： > > `/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml` > > 发生了此错误。你遇到过吗 > > > /root/.profile: line 10: /.cargo/env: No such file or directory > > [2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up) > > [2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6 > > [2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true > > [2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true > > [2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24 > > [2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN > > [2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt > > [2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports > > [2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started. > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true, > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [ > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal" > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ] > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL", > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr" > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > [2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}} > > [2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > *** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') *** > > *** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: *** > > [2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > [2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > [2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > [2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > [2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > @ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv > > /3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449 > > @ 000000000004251f (unknown) > > @ 00000000000969fc pthread_kill > > @ 0000000000042475 raise > > @ 00000000000287f2 abort > > @ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE > > /3fs/third_party/folly/folly/logging/LogCategory.cpp:71 > > @ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv > > /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190 > > @ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo > > /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222 > > @ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token > > /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49 > > @ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv > > /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74 > > -> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc > > @ 00000000000dc252 (unknown) > > @ 0000000000094ac2 (unknown) > > @ 0000000000125a03 clone > > Aborted (core dumped) 你网络硬件不支持，没有IB网卡吧， ```bash docker run --privileged --device=/dev/infiniband:/dev/infiniband -it 3fs-build ``` 注意设备
bsakaguchi commented 2026-10-26 09:44:02 +00:00
Author
Owner
Copy link

@shuiyihang commented on GitHub (Mar 4, 2025):

感谢您的指导，我在容器中使用了编译好的程序，如下所示：
/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml
发生了此错误。你遇到过吗

/root/.profile: line 10: /.cargo/env: No such file or directory
[2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up)
[2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6
[2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true
[2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true
[2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24
[2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt
[2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports
[2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started.
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true,
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ]
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}}
[2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
*** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') ***
*** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: ***
[2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
@ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv
/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449
@ 000000000004251f (unknown)
@ 00000000000969fc pthread_kill
@ 0000000000042475 raise
@ 00000000000287f2 abort
@ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE
/3fs/third_party/folly/folly/logging/LogCategory.cpp:71
@ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190
@ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222
@ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token
/3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49
@ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74
-> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc
@ 00000000000dc252 (unknown)
@ 0000000000094ac2 (unknown)
@ 0000000000125a03 clone
Aborted (core dumped)

你网络硬件不支持，没有IB网卡吧，

docker run --privileged --device=/dev/infiniband:/dev/infiniband -it 3fs-build
注意设备

这个问题刚刚解决了，是我的clickhouse没有配置正确。
我在Step 4: Mgmtd service遇到了新的问题，使用/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 大约十几分钟终端一直被卡着，无法完成，您知道这是什么情况吗

<!-- gh-comment-id:2696866538 --> @shuiyihang commented on GitHub (Mar 4, 2025): > > 感谢您的指导，我在容器中使用了编译好的程序，如下所示： > > `/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml` > > 发生了此错误。你遇到过吗 > > > /root/.profile: line 10: /.cargo/env: No such file or directory > > > [2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up) > > > [2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6 > > > [2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true > > > [2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true > > > [2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24 > > > [2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN > > > [2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt > > > [2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports > > > [2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started. > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true, > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [ > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal" > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ] > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL", > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr" > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > [2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}} > > > [2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > *** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') *** > > > *** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: *** > > > [2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > [2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > [2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > [2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > [2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > @ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv > > > /3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449 > > > @ 000000000004251f (unknown) > > > @ 00000000000969fc pthread_kill > > > @ 0000000000042475 raise > > > @ 00000000000287f2 abort > > > @ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE > > > /3fs/third_party/folly/folly/logging/LogCategory.cpp:71 > > > @ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv > > > /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190 > > > @ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo > > > /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222 > > > @ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token > > > /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49 > > > @ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv > > > /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74 > > > -> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc > > > @ 00000000000dc252 (unknown) > > > @ 0000000000094ac2 (unknown) > > > @ 0000000000125a03 clone > > > Aborted (core dumped) > > 你网络硬件不支持，没有IB网卡吧， > > docker run --privileged --device=/dev/infiniband:/dev/infiniband -it 3fs-build > 注意设备 这个问题刚刚解决了，是我的clickhouse没有配置正确。 我在Step 4: Mgmtd service遇到了新的问题，使用`/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16"` 大约十几分钟终端一直被卡着，无法完成，您知道这是什么情况吗
bsakaguchi commented 2026-10-26 09:44:03 +00:00
Author
Owner
Copy link

@szukeymaker commented on GitHub (Mar 5, 2025):

感谢您的指导，我在容器中使用了编译好的程序，如下所示：
/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml
发生了此错误。你遇到过吗

/root/.profile: line 10: /.cargo/env: No such file or directory
[2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up)
[2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6
[2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true
[2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true
[2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24
[2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt
[2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports
[2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started.
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true,
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ]
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}}
[2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
*** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') ***
*** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: ***
[2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
@ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv
/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449
@ 000000000004251f (unknown)
@ 00000000000969fc pthread_kill
@ 0000000000042475 raise
@ 00000000000287f2 abort
@ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE
/3fs/third_party/folly/folly/logging/LogCategory.cpp:71
@ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190
@ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222
@ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token
/3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49
@ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74
-> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc
@ 00000000000dc252 (unknown)
@ 0000000000094ac2 (unknown)
@ 0000000000125a03 clone
Aborted (core dumped)

你网络硬件不支持，没有IB网卡吧，
docker run --privileged --device=/dev/infiniband:/dev/infiniband -it 3fs-build
注意设备

这个问题刚刚解决了，是我的clickhouse没有配置正确。 我在Step 4: Mgmtd service遇到了新的问题，使用/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 大约十几分钟终端一直被卡着，无法完成，您知道这是什么情况吗

systemctl start foundationdb

<!-- gh-comment-id:2699856246 --> @szukeymaker commented on GitHub (Mar 5, 2025): > > > 感谢您的指导，我在容器中使用了编译好的程序，如下所示： > > > `/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml` > > > 发生了此错误。你遇到过吗 > > > > /root/.profile: line 10: /.cargo/env: No such file or directory > > > > [2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up) > > > > [2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6 > > > > [2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true > > > > [2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true > > > > [2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24 > > > > [2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN > > > > [2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt > > > > [2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports > > > > [2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started. > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true, > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [ > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal" > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ] > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL", > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr" > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > [2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}} > > > > [2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > *** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') *** > > > > *** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: *** > > > > [2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > [2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > [2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > [2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > [2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > @ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv > > > > /3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449 > > > > @ 000000000004251f (unknown) > > > > @ 00000000000969fc pthread_kill > > > > @ 0000000000042475 raise > > > > @ 00000000000287f2 abort > > > > @ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE > > > > /3fs/third_party/folly/folly/logging/LogCategory.cpp:71 > > > > @ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv > > > > /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190 > > > > @ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo > > > > /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222 > > > > @ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token > > > > /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49 > > > > @ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv > > > > /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74 > > > > -> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc > > > > @ 00000000000dc252 (unknown) > > > > @ 0000000000094ac2 (unknown) > > > > @ 0000000000125a03 clone > > > > Aborted (core dumped) > > > > > > 你网络硬件不支持，没有IB网卡吧， > > docker run --privileged --device=/dev/infiniband:/dev/infiniband -it 3fs-build > > 注意设备 > > 这个问题刚刚解决了，是我的clickhouse没有配置正确。 我在Step 4: Mgmtd service遇到了新的问题，使用`/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16"` 大约十几分钟终端一直被卡着，无法完成，您知道这是什么情况吗 systemctl start foundationdb
bsakaguchi commented 2026-10-26 09:44:03 +00:00
Author
Owner
Copy link

@shuiyihang commented on GitHub (Mar 5, 2025):

感谢您的指导，我在容器中使用了编译好的程序，如下所示：
/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml
发生了此错误。你遇到过吗

/root/.profile: line 10: /.cargo/env: No such file or directory
[2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up)
[2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6
[2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true
[2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true
[2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24
[2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt
[2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports
[2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started.
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true,
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ]
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] },
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": {
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL",
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr"
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }
[2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}}
[2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
*** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') ***
*** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: ***
[2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
[2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12)
@ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv
/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449
@ 000000000004251f (unknown)
@ 00000000000969fc pthread_kill
@ 0000000000042475 raise
@ 00000000000287f2 abort
@ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE
/3fs/third_party/folly/folly/logging/LogCategory.cpp:71
@ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190
@ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo
/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222
@ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token
/3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49
@ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74
-> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc
@ 00000000000dc252 (unknown)
@ 0000000000094ac2 (unknown)
@ 0000000000125a03 clone
Aborted (core dumped)

你网络硬件不支持，没有IB网卡吧，
docker run --privileged --device=/dev/infiniband:/dev/infiniband -it 3fs-build
注意设备

这个问题刚刚解决了，是我的clickhouse没有配置正确。 我在Step 4: Mgmtd service遇到了新的问题，使用/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 大约十几分钟终端一直被卡着，无法完成，您知道这是什么情况吗

systemctl start foundationdb

thanks,it's work!
service foundationdb start

<!-- gh-comment-id:2700028328 --> @shuiyihang commented on GitHub (Mar 5, 2025): > > > > 感谢您的指导，我在容器中使用了编译好的程序，如下所示： > > > > `/opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml` > > > > 发生了此错误。你遇到过吗 > > > > > /root/.profile: line 10: /.cargo/env: No such file or directory > > > > > [2025-03-04T14:18:12.322834406+08:00 monitor_collect: 1741 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> ibs6 (Up) > > > > > [2025-03-04T14:18:12.322919759+08:00 monitor_collect: 1741 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => ibs6 > > > > > [2025-03-04T14:18:12.323383214+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ens16f0, addr 192.168.1.235/18, subnet 192.168.0.0/18, up true > > > > > [2025-03-04T14:18:12.323396003+08:00 monitor_collect: 1741 IfAddrs.h:102 INFO] Get ifaddr of ibs6, addr 10.200.1.235/24, subnet 10.200.1.0/24, up true > > > > > [2025-03-04T14:18:12.327728027+08:00 monitor_collect: 1741 IBDevice.cc:396 INFO] IBDevice mlx5_0:1 IP addrs 10.200.1.235/24 > > > > > [2025-03-04T14:18:12.327740602+08:00 monitor_collect: 1741 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN > > > > > [2025-03-04T14:18:12.327747847+08:00 monitor_collect: 1741 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer INFINIBAND, addrs 10.200.1.235/24, zones UNKNOWN, RoCE v2 GID std::nullopt > > > > > [2025-03-04T14:18:12.327753550+08:00 monitor_collect: 1741 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports > > > > > [2025-03-04T14:18:12.329104934+08:00 IBManager: 1776 EventLoop.cc:116 INFO] EventLoop::loop() started. > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] Folly log json configure: { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "categories": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ".": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "INFO", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "inherit": true, > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "propagate": "NONE", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": [ > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal" > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] ] > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "handlers": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "normal": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main.log", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "true", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "err": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "file", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "ERR", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "path": "/var/log/3fs/monitor_collector_main-err.log", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "async": "false", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate": "true", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_files": "10", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "max_file_size": "104857600", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "rotate_on_open": "false" > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] }, > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "fatal": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "type": "stream", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "options": { > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "level": "FATAL", > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] "stream": "stderr" > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > > [2025-03-04T14:18:12.329226549+08:00 monitor_collect: 1741 LogConfig.cc:96 INFO] } > > > > > [2025-03-04T14:18:12.329280666+08:00 monitor_collect: 1741 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}} > > > > > [2025-03-04T14:18:12.373070739+08:00 monitor_collect: 1798 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > > *** Aborted at 1741069092 (Unix time, try 'date -d @1741069092') *** > > > > > *** Signal 6 (SIGABRT) (0x6cd) received by PID 1741 (pthread TID 0x7f89a57fa640) (linux TID 1798) (maybe from PID 1741, UID 0) (code: -6), stack trace: *** > > > > > [2025-03-04T14:18:12.375834757+08:00 monitor_collect: 1799 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > > [2025-03-04T14:18:12.376456642+08:00 monitor_collect: 1800 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > > [2025-03-04T14:18:12.377777684+08:00 monitor_collect: 1801 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > > [2025-03-04T14:18:12.380235650+08:00 monitor_collect: 1802 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > > [2025-03-04T14:18:12.381910863+08:00 monitor_collect: 1803 MonitorCollectorOperator.cc:49 FATAL] Initializing reporter failed. MonitorInitFailed(12) > > > > > @ 000000000043179f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv > > > > > /3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449 > > > > > @ 000000000004251f (unknown) > > > > > @ 00000000000969fc pthread_kill > > > > > @ 0000000000042475 raise > > > > > @ 00000000000287f2 abort > > > > > @ 0000000000499460 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE > > > > > /3fs/third_party/folly/folly/logging/LogCategory.cpp:71 > > > > > @ 0000000000498916 _ZN5folly18LogStreamProcessor6logNowEv > > > > > /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190 > > > > > @ 0000000000498b69 _ZN5folly16LogStreamVoidifyILb1EEanERSo > > > > > /3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222 > > > > > @ 000000000019ded1 _ZN5hf3fs7monitor24MonitorCollectorOperator14connThreadFuncESt10stop_token > > > > > /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc:49 > > > > > @ 000000000019f428 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN5hf3fs7monitor24MonitorCollectorOperatorEFvSt10stop_tokenEJPS6_EES7_EEEEE6_M_runEv > > > > > /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:74 > > > > > -> /3fs/src/monitor_collector/service/MonitorCollectorOperator.cc > > > > > @ 00000000000dc252 (unknown) > > > > > @ 0000000000094ac2 (unknown) > > > > > @ 0000000000125a03 clone > > > > > Aborted (core dumped) > > > > > > > > > 你网络硬件不支持，没有IB网卡吧， > > > docker run --privileged --device=/dev/infiniband:/dev/infiniband -it 3fs-build > > > 注意设备 > > > > > > 这个问题刚刚解决了，是我的clickhouse没有配置正确。 我在Step 4: Mgmtd service遇到了新的问题，使用`/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16"` 大约十几分钟终端一直被卡着，无法完成，您知道这是什么情况吗 > > systemctl start foundationdb thanks,it's work! `service foundationdb start`
bsakaguchi commented 2026-10-26 09:44:04 +00:00
Author
Owner
Copy link

@photoszzt commented on GitHub (Mar 6, 2025):

Maybe it's time to package it as daemonset and helm charts for Kubernetes.

<!-- gh-comment-id:2704615551 --> @photoszzt commented on GitHub (Mar 6, 2025): Maybe it's time to package it as daemonset and helm charts for Kubernetes.
bsakaguchi commented 2026-10-26 09:44:04 +00:00
Author
Owner
Copy link

@xiaoyaoyouyue commented on GitHub (Mar 7, 2025):

请教下，当我使用docker-compose.yaml 和Dockerfile 生成了3fs的镜像以及容器了以后，是否可以复制多分，然后来分别构件meta，storage，management，client 几个容器，来分别承担对应功能？

<!-- gh-comment-id:2705360046 --> @xiaoyaoyouyue commented on GitHub (Mar 7, 2025): 请教下，当我使用docker-compose.yaml 和Dockerfile 生成了3fs的镜像以及容器了以后，是否可以复制多分，然后来分别构件meta，storage，management，client 几个容器，来分别承担对应功能？
bsakaguchi commented 2026-10-26 09:44:04 +00:00
Author
Owner
Copy link

@Icedroid commented on GitHub (Mar 14, 2025):

@shuiyihang 大佬，这个问题你有遇到过吗？

root@localhost:/opt/3fs# /opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml
[2025-03-14T13:08:16.512822042+00:00 monitor_collect:  156 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> Bond0 (Up)
[2025-03-14T13:08:16.512822042+00:00 monitor_collect:  156 IBDevice.cc:169 INFO] mlx5_1 port 1 ==> Bond0 (Up)
[2025-03-14T13:08:16.512822042+00:00 monitor_collect:  156 IBDevice.cc:169 INFO] mlx5_2 port 1 ==> Bond0 (Up)
[2025-03-14T13:08:16.512822042+00:00 monitor_collect:  156 IBDevice.cc:169 INFO] mlx5_3 port 1 ==> Bond0 (Up)
[2025-03-14T13:08:16.512876530+00:00 monitor_collect:  156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => Bond0
[2025-03-14T13:08:16.512880240+00:00 monitor_collect:  156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_1 => Bond0
[2025-03-14T13:08:16.512882982+00:00 monitor_collect:  156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_2 => Bond0
[2025-03-14T13:08:16.512887535+00:00 monitor_collect:  156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_3 => Bond0
[2025-03-14T13:08:16.513531239+00:00 monitor_collect:  156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.2503, addr 192.168.3.25/24, subnet 192.168.3.0/24, up true
[2025-03-14T13:08:16.513539473+00:00 monitor_collect:  156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.211, addr 192.168.211.25/24, subnet 192.168.211.0/24, up true
[2025-03-14T13:08:16.513545590+00:00 monitor_collect:  156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.202, addr 192.168.202.25/24, subnet 192.168.202.0/24, up true
[2025-03-14T13:08:16.513550796+00:00 monitor_collect:  156 IfAddrs.h:102 INFO] Get ifaddr of Bond1.193, addr 10.129.185.25/24, subnet 10.129.185.0/24, up true
[2025-03-14T13:08:16.517824232+00:00 monitor_collect:  156 IBDevice.cc:386 WARNING] IfAddr of mlx5_0:1 -> Bond0 not found, maybe running in container!
[2025-03-14T13:08:16.517831953+00:00 monitor_collect:  156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-14T13:08:16.517864933+00:00 monitor_collect:  156 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0:
0:0:ff:ff:c0:a8:3:19
[2025-03-14T13:08:16.517871053+00:00 monitor_collect:  156 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports
[2025-03-14T13:08:16.520906330+00:00 monitor_collect:  156 IBDevice.cc:386 WARNING] IfAddr of mlx5_1:1 -> Bond0 not found, maybe running in container!
[2025-03-14T13:08:16.520910725+00:00 monitor_collect:  156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_1:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-14T13:08:16.520927516+00:00 monitor_collect:  156 IBDevice.cc:367 INFO] IBDevice mlx5_1 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0:
0:0:ff:ff:c0:a8:3:19
[2025-03-14T13:08:16.520932036+00:00 monitor_collect:  156 IBDevice.cc:256 INFO] IBDevice add mlx5_1, id 1, 1 available ports
[2025-03-14T13:08:16.524056057+00:00 monitor_collect:  156 IBDevice.cc:386 WARNING] IfAddr of mlx5_2:1 -> Bond0 not found, maybe running in container!
[2025-03-14T13:08:16.524060248+00:00 monitor_collect:  156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_2:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-14T13:08:16.524076743+00:00 monitor_collect:  156 IBDevice.cc:367 INFO] IBDevice mlx5_2 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0:
0:0:ff:ff:c0:a8:3:19
[2025-03-14T13:08:16.524080967+00:00 monitor_collect:  156 IBDevice.cc:256 INFO] IBDevice add mlx5_2, id 2, 1 available ports
[2025-03-14T13:08:16.527724009+00:00 monitor_collect:  156 IBDevice.cc:386 WARNING] IfAddr of mlx5_3:1 -> Bond0 not found, maybe running in container!
[2025-03-14T13:08:16.527728256+00:00 monitor_collect:  156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_3:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-14T13:08:16.527744727+00:00 monitor_collect:  156 IBDevice.cc:367 INFO] IBDevice mlx5_3 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0:
0:0:ff:ff:c0:a8:3:19
[2025-03-14T13:08:16.527748674+00:00 monitor_collect:  156 IBDevice.cc:256 INFO] IBDevice add mlx5_3, id 3, 1 available ports
[2025-03-14T13:08:16.529142628+00:00 IBManager:  210 EventLoop.cc:116 INFO] EventLoop::loop() started.
[2025-03-14T13:08:16.529273685+00:00 monitor_collect:  156 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["norm
al","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"10485
7600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_f
ile_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}}
[2025-03-14T13:08:16.559326006+00:00 monitor_collect:  156 OnePhaseApplication.h:101 FATAL] Setup server failed: RPC::ListenFailed(2011)
*** Aborted at 1741957696 (Unix time, try 'date -d @1741957696') ***
*** Signal 6 (SIGABRT) (0x9c) received by PID 156 (pthread TID 0x7f5f45088600) (linux TID 156) (maybe from PID 156, UID 0) (code: -6), stack trace: ***
    @ 00000000004e9e9f (unknown)
    @ 000000000004251f (unknown)
    @ 00000000000969fc pthread_kill
    @ 0000000000042475 raise
    @ 00000000000287f2 abort
    @ 0000000000551b60 (unknown)
    @ 0000000000551016 (unknown)
    @ 0000000000551269 (unknown)
    @ 0000000000240491 (unknown)
    @ 0000000000262d46 (unknown)
    @ 0000000000029d8f (unknown)
    @ 0000000000029e3f __libc_start_main
    @ 000000000018dfe4 (unknown)
Aborted (core dumped)
<!-- gh-comment-id:2724657467 --> @Icedroid commented on GitHub (Mar 14, 2025): @shuiyihang 大佬，这个问题你有遇到过吗？ ``` root@localhost:/opt/3fs# /opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml [2025-03-14T13:08:16.512822042+00:00 monitor_collect: 156 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> Bond0 (Up) [2025-03-14T13:08:16.512822042+00:00 monitor_collect: 156 IBDevice.cc:169 INFO] mlx5_1 port 1 ==> Bond0 (Up) [2025-03-14T13:08:16.512822042+00:00 monitor_collect: 156 IBDevice.cc:169 INFO] mlx5_2 port 1 ==> Bond0 (Up) [2025-03-14T13:08:16.512822042+00:00 monitor_collect: 156 IBDevice.cc:169 INFO] mlx5_3 port 1 ==> Bond0 (Up) [2025-03-14T13:08:16.512876530+00:00 monitor_collect: 156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => Bond0 [2025-03-14T13:08:16.512880240+00:00 monitor_collect: 156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_1 => Bond0 [2025-03-14T13:08:16.512882982+00:00 monitor_collect: 156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_2 => Bond0 [2025-03-14T13:08:16.512887535+00:00 monitor_collect: 156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_3 => Bond0 [2025-03-14T13:08:16.513531239+00:00 monitor_collect: 156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.2503, addr 192.168.3.25/24, subnet 192.168.3.0/24, up true [2025-03-14T13:08:16.513539473+00:00 monitor_collect: 156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.211, addr 192.168.211.25/24, subnet 192.168.211.0/24, up true [2025-03-14T13:08:16.513545590+00:00 monitor_collect: 156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.202, addr 192.168.202.25/24, subnet 192.168.202.0/24, up true [2025-03-14T13:08:16.513550796+00:00 monitor_collect: 156 IfAddrs.h:102 INFO] Get ifaddr of Bond1.193, addr 10.129.185.25/24, subnet 10.129.185.0/24, up true [2025-03-14T13:08:16.517824232+00:00 monitor_collect: 156 IBDevice.cc:386 WARNING] IfAddr of mlx5_0:1 -> Bond0 not found, maybe running in container! [2025-03-14T13:08:16.517831953+00:00 monitor_collect: 156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN [2025-03-14T13:08:16.517864933+00:00 monitor_collect: 156 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0: 0:0:ff:ff:c0:a8:3:19 [2025-03-14T13:08:16.517871053+00:00 monitor_collect: 156 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports [2025-03-14T13:08:16.520906330+00:00 monitor_collect: 156 IBDevice.cc:386 WARNING] IfAddr of mlx5_1:1 -> Bond0 not found, maybe running in container! [2025-03-14T13:08:16.520910725+00:00 monitor_collect: 156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_1:1 can't set zone by IP, fallback to UNKNOWN [2025-03-14T13:08:16.520927516+00:00 monitor_collect: 156 IBDevice.cc:367 INFO] IBDevice mlx5_1 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0: 0:0:ff:ff:c0:a8:3:19 [2025-03-14T13:08:16.520932036+00:00 monitor_collect: 156 IBDevice.cc:256 INFO] IBDevice add mlx5_1, id 1, 1 available ports [2025-03-14T13:08:16.524056057+00:00 monitor_collect: 156 IBDevice.cc:386 WARNING] IfAddr of mlx5_2:1 -> Bond0 not found, maybe running in container! [2025-03-14T13:08:16.524060248+00:00 monitor_collect: 156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_2:1 can't set zone by IP, fallback to UNKNOWN [2025-03-14T13:08:16.524076743+00:00 monitor_collect: 156 IBDevice.cc:367 INFO] IBDevice mlx5_2 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0: 0:0:ff:ff:c0:a8:3:19 [2025-03-14T13:08:16.524080967+00:00 monitor_collect: 156 IBDevice.cc:256 INFO] IBDevice add mlx5_2, id 2, 1 available ports [2025-03-14T13:08:16.527724009+00:00 monitor_collect: 156 IBDevice.cc:386 WARNING] IfAddr of mlx5_3:1 -> Bond0 not found, maybe running in container! [2025-03-14T13:08:16.527728256+00:00 monitor_collect: 156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_3:1 can't set zone by IP, fallback to UNKNOWN [2025-03-14T13:08:16.527744727+00:00 monitor_collect: 156 IBDevice.cc:367 INFO] IBDevice mlx5_3 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0: 0:0:ff:ff:c0:a8:3:19 [2025-03-14T13:08:16.527748674+00:00 monitor_collect: 156 IBDevice.cc:256 INFO] IBDevice add mlx5_3, id 3, 1 available ports [2025-03-14T13:08:16.529142628+00:00 IBManager: 210 EventLoop.cc:116 INFO] EventLoop::loop() started. [2025-03-14T13:08:16.529273685+00:00 monitor_collect: 156 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["norm al","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"10485 7600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_f ile_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}} [2025-03-14T13:08:16.559326006+00:00 monitor_collect: 156 OnePhaseApplication.h:101 FATAL] Setup server failed: RPC::ListenFailed(2011) *** Aborted at 1741957696 (Unix time, try 'date -d @1741957696') *** *** Signal 6 (SIGABRT) (0x9c) received by PID 156 (pthread TID 0x7f5f45088600) (linux TID 156) (maybe from PID 156, UID 0) (code: -6), stack trace: *** @ 00000000004e9e9f (unknown) @ 000000000004251f (unknown) @ 00000000000969fc pthread_kill @ 0000000000042475 raise @ 00000000000287f2 abort @ 0000000000551b60 (unknown) @ 0000000000551016 (unknown) @ 0000000000551269 (unknown) @ 0000000000240491 (unknown) @ 0000000000262d46 (unknown) @ 0000000000029d8f (unknown) @ 0000000000029e3f __libc_start_main @ 000000000018dfe4 (unknown) Aborted (core dumped) ```
bsakaguchi commented 2026-10-26 09:44:05 +00:00
Author
Owner
Copy link

@Icedroid commented on GitHub (Mar 14, 2025):

@shuiyihang 大佬，这个问题你有遇到过吗？

root@localhost:/opt/3fs# /opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml
[2025-03-14T13:08:16.512822042+00:00 monitor_collect:  156 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> Bond0 (Up)
[2025-03-14T13:08:16.512822042+00:00 monitor_collect:  156 IBDevice.cc:169 INFO] mlx5_1 port 1 ==> Bond0 (Up)
[2025-03-14T13:08:16.512822042+00:00 monitor_collect:  156 IBDevice.cc:169 INFO] mlx5_2 port 1 ==> Bond0 (Up)
[2025-03-14T13:08:16.512822042+00:00 monitor_collect:  156 IBDevice.cc:169 INFO] mlx5_3 port 1 ==> Bond0 (Up)
[2025-03-14T13:08:16.512876530+00:00 monitor_collect:  156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => Bond0
[2025-03-14T13:08:16.512880240+00:00 monitor_collect:  156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_1 => Bond0
[2025-03-14T13:08:16.512882982+00:00 monitor_collect:  156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_2 => Bond0
[2025-03-14T13:08:16.512887535+00:00 monitor_collect:  156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_3 => Bond0
[2025-03-14T13:08:16.513531239+00:00 monitor_collect:  156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.2503, addr 192.168.3.25/24, subnet 192.168.3.0/24, up true
[2025-03-14T13:08:16.513539473+00:00 monitor_collect:  156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.211, addr 192.168.211.25/24, subnet 192.168.211.0/24, up true
[2025-03-14T13:08:16.513545590+00:00 monitor_collect:  156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.202, addr 192.168.202.25/24, subnet 192.168.202.0/24, up true
[2025-03-14T13:08:16.513550796+00:00 monitor_collect:  156 IfAddrs.h:102 INFO] Get ifaddr of Bond1.193, addr 10.129.185.25/24, subnet 10.129.185.0/24, up true
[2025-03-14T13:08:16.517824232+00:00 monitor_collect:  156 IBDevice.cc:386 WARNING] IfAddr of mlx5_0:1 -> Bond0 not found, maybe running in container!
[2025-03-14T13:08:16.517831953+00:00 monitor_collect:  156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-14T13:08:16.517864933+00:00 monitor_collect:  156 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0:
0:0:ff:ff:c0:a8:3:19
[2025-03-14T13:08:16.517871053+00:00 monitor_collect:  156 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports
[2025-03-14T13:08:16.520906330+00:00 monitor_collect:  156 IBDevice.cc:386 WARNING] IfAddr of mlx5_1:1 -> Bond0 not found, maybe running in container!
[2025-03-14T13:08:16.520910725+00:00 monitor_collect:  156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_1:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-14T13:08:16.520927516+00:00 monitor_collect:  156 IBDevice.cc:367 INFO] IBDevice mlx5_1 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0:
0:0:ff:ff:c0:a8:3:19
[2025-03-14T13:08:16.520932036+00:00 monitor_collect:  156 IBDevice.cc:256 INFO] IBDevice add mlx5_1, id 1, 1 available ports
[2025-03-14T13:08:16.524056057+00:00 monitor_collect:  156 IBDevice.cc:386 WARNING] IfAddr of mlx5_2:1 -> Bond0 not found, maybe running in container!
[2025-03-14T13:08:16.524060248+00:00 monitor_collect:  156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_2:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-14T13:08:16.524076743+00:00 monitor_collect:  156 IBDevice.cc:367 INFO] IBDevice mlx5_2 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0:
0:0:ff:ff:c0:a8:3:19
[2025-03-14T13:08:16.524080967+00:00 monitor_collect:  156 IBDevice.cc:256 INFO] IBDevice add mlx5_2, id 2, 1 available ports
[2025-03-14T13:08:16.527724009+00:00 monitor_collect:  156 IBDevice.cc:386 WARNING] IfAddr of mlx5_3:1 -> Bond0 not found, maybe running in container!
[2025-03-14T13:08:16.527728256+00:00 monitor_collect:  156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_3:1 can't set zone by IP, fallback to UNKNOWN
[2025-03-14T13:08:16.527744727+00:00 monitor_collect:  156 IBDevice.cc:367 INFO] IBDevice mlx5_3 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0:
0:0:ff:ff:c0:a8:3:19
[2025-03-14T13:08:16.527748674+00:00 monitor_collect:  156 IBDevice.cc:256 INFO] IBDevice add mlx5_3, id 3, 1 available ports
[2025-03-14T13:08:16.529142628+00:00 IBManager:  210 EventLoop.cc:116 INFO] EventLoop::loop() started.
[2025-03-14T13:08:16.529273685+00:00 monitor_collect:  156 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["norm
al","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"10485
7600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_f
ile_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}}
[2025-03-14T13:08:16.559326006+00:00 monitor_collect:  156 OnePhaseApplication.h:101 FATAL] Setup server failed: RPC::ListenFailed(2011)
*** Aborted at 1741957696 (Unix time, try 'date -d @1741957696') ***
*** Signal 6 (SIGABRT) (0x9c) received by PID 156 (pthread TID 0x7f5f45088600) (linux TID 156) (maybe from PID 156, UID 0) (code: -6), stack trace: ***
    @ 00000000004e9e9f (unknown)
    @ 000000000004251f (unknown)
    @ 00000000000969fc pthread_kill
    @ 0000000000042475 raise
    @ 00000000000287f2 abort
    @ 0000000000551b60 (unknown)
    @ 0000000000551016 (unknown)
    @ 0000000000551269 (unknown)
    @ 0000000000240491 (unknown)
    @ 0000000000262d46 (unknown)
    @ 0000000000029d8f (unknown)
    @ 0000000000029e3f __libc_start_main
    @ 000000000018dfe4 (unknown)
Aborted (core dumped)

这个解决，要搜索源码，将有fbs.startsWith("bond")的地方，同样加个判断 fbs.startsWith("Bond")

<!-- gh-comment-id:2725124942 --> @Icedroid commented on GitHub (Mar 14, 2025): > [@shuiyihang](https://github.com/shuiyihang) 大佬，这个问题你有遇到过吗？ > > ``` > root@localhost:/opt/3fs# /opt/3fs/bin/monitor_collector_main --cfg /opt/3fs/etc/monitor_collector_main.toml > [2025-03-14T13:08:16.512822042+00:00 monitor_collect: 156 IBDevice.cc:169 INFO] ibdev2netdev: mlx5_0 port 1 ==> Bond0 (Up) > [2025-03-14T13:08:16.512822042+00:00 monitor_collect: 156 IBDevice.cc:169 INFO] mlx5_1 port 1 ==> Bond0 (Up) > [2025-03-14T13:08:16.512822042+00:00 monitor_collect: 156 IBDevice.cc:169 INFO] mlx5_2 port 1 ==> Bond0 (Up) > [2025-03-14T13:08:16.512822042+00:00 monitor_collect: 156 IBDevice.cc:169 INFO] mlx5_3 port 1 ==> Bond0 (Up) > [2025-03-14T13:08:16.512876530+00:00 monitor_collect: 156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_0 => Bond0 > [2025-03-14T13:08:16.512880240+00:00 monitor_collect: 156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_1 => Bond0 > [2025-03-14T13:08:16.512882982+00:00 monitor_collect: 156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_2 => Bond0 > [2025-03-14T13:08:16.512887535+00:00 monitor_collect: 156 IBDevice.cc:186 INFO] ibdev2netdev parsed: mlx5_3 => Bond0 > [2025-03-14T13:08:16.513531239+00:00 monitor_collect: 156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.2503, addr 192.168.3.25/24, subnet 192.168.3.0/24, up true > [2025-03-14T13:08:16.513539473+00:00 monitor_collect: 156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.211, addr 192.168.211.25/24, subnet 192.168.211.0/24, up true > [2025-03-14T13:08:16.513545590+00:00 monitor_collect: 156 IfAddrs.h:102 INFO] Get ifaddr of Bond0.202, addr 192.168.202.25/24, subnet 192.168.202.0/24, up true > [2025-03-14T13:08:16.513550796+00:00 monitor_collect: 156 IfAddrs.h:102 INFO] Get ifaddr of Bond1.193, addr 10.129.185.25/24, subnet 10.129.185.0/24, up true > [2025-03-14T13:08:16.517824232+00:00 monitor_collect: 156 IBDevice.cc:386 WARNING] IfAddr of mlx5_0:1 -> Bond0 not found, maybe running in container! > [2025-03-14T13:08:16.517831953+00:00 monitor_collect: 156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_0:1 can't set zone by IP, fallback to UNKNOWN > [2025-03-14T13:08:16.517864933+00:00 monitor_collect: 156 IBDevice.cc:367 INFO] IBDevice mlx5_0 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0: > 0:0:ff:ff:c0:a8:3:19 > [2025-03-14T13:08:16.517871053+00:00 monitor_collect: 156 IBDevice.cc:256 INFO] IBDevice add mlx5_0, id 0, 1 available ports > [2025-03-14T13:08:16.520906330+00:00 monitor_collect: 156 IBDevice.cc:386 WARNING] IfAddr of mlx5_1:1 -> Bond0 not found, maybe running in container! > [2025-03-14T13:08:16.520910725+00:00 monitor_collect: 156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_1:1 can't set zone by IP, fallback to UNKNOWN > [2025-03-14T13:08:16.520927516+00:00 monitor_collect: 156 IBDevice.cc:367 INFO] IBDevice mlx5_1 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0: > 0:0:ff:ff:c0:a8:3:19 > [2025-03-14T13:08:16.520932036+00:00 monitor_collect: 156 IBDevice.cc:256 INFO] IBDevice add mlx5_1, id 1, 1 available ports > [2025-03-14T13:08:16.524056057+00:00 monitor_collect: 156 IBDevice.cc:386 WARNING] IfAddr of mlx5_2:1 -> Bond0 not found, maybe running in container! > [2025-03-14T13:08:16.524060248+00:00 monitor_collect: 156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_2:1 can't set zone by IP, fallback to UNKNOWN > [2025-03-14T13:08:16.524076743+00:00 monitor_collect: 156 IBDevice.cc:367 INFO] IBDevice mlx5_2 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0: > 0:0:ff:ff:c0:a8:3:19 > [2025-03-14T13:08:16.524080967+00:00 monitor_collect: 156 IBDevice.cc:256 INFO] IBDevice add mlx5_2, id 2, 1 available ports > [2025-03-14T13:08:16.527724009+00:00 monitor_collect: 156 IBDevice.cc:386 WARNING] IfAddr of mlx5_3:1 -> Bond0 not found, maybe running in container! > [2025-03-14T13:08:16.527728256+00:00 monitor_collect: 156 IBDevice.cc:441 CRITICAL] IBDevice mlx5_3:1 can't set zone by IP, fallback to UNKNOWN > [2025-03-14T13:08:16.527744727+00:00 monitor_collect: 156 IBDevice.cc:367 INFO] IBDevice mlx5_3 add active port 1, linklayer ETHERNET, addrs , zones UNKNOWN, RoCE v2 GID 0:0:0:0:0:0:0:0: > 0:0:ff:ff:c0:a8:3:19 > [2025-03-14T13:08:16.527748674+00:00 monitor_collect: 156 IBDevice.cc:256 INFO] IBDevice add mlx5_3, id 3, 1 available ports > [2025-03-14T13:08:16.529142628+00:00 IBManager: 210 EventLoop.cc:116 INFO] EventLoop::loop() started. > [2025-03-14T13:08:16.529273685+00:00 monitor_collect: 156 OnePhaseApplication.h:87 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["norm > al","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/monitor_collector_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"10485 > 7600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/monitor_collector_main-err.log","async":"false","rotate":"true","max_files":"10","max_f > ile_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}} > [2025-03-14T13:08:16.559326006+00:00 monitor_collect: 156 OnePhaseApplication.h:101 FATAL] Setup server failed: RPC::ListenFailed(2011) > *** Aborted at 1741957696 (Unix time, try 'date -d @1741957696') *** > *** Signal 6 (SIGABRT) (0x9c) received by PID 156 (pthread TID 0x7f5f45088600) (linux TID 156) (maybe from PID 156, UID 0) (code: -6), stack trace: *** > @ 00000000004e9e9f (unknown) > @ 000000000004251f (unknown) > @ 00000000000969fc pthread_kill > @ 0000000000042475 raise > @ 00000000000287f2 abort > @ 0000000000551b60 (unknown) > @ 0000000000551016 (unknown) > @ 0000000000551269 (unknown) > @ 0000000000240491 (unknown) > @ 0000000000262d46 (unknown) > @ 0000000000029d8f (unknown) > @ 0000000000029e3f __libc_start_main > @ 000000000018dfe4 (unknown) > Aborted (core dumped) > ``` 这个解决，要搜索源码，将有fbs.startsWith("bond")的地方，同样加个判断 fbs.startsWith("Bond")
bsakaguchi commented 2026-10-26 09:44:05 +00:00
Author
Owner
Copy link

@Icedroid commented on GitHub (Mar 14, 2025):

我也发现这个一直卡住是什么原因，我foundationdb服务是启动的
/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16"

<!-- gh-comment-id:2725127250 --> @Icedroid commented on GitHub (Mar 14, 2025): 我也发现这个一直卡住是什么原因，我foundationdb服务是启动的 /opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16"
bsakaguchi commented 2026-10-26 09:44:05 +00:00
Author
Owner
Copy link

@gongweicher commented on GitHub (Apr 21, 2025):

我也发现这个一直卡住是什么原因，我foundationdb服务是启动的 /opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16"

hello, 我遇到了同样的问题。foundationdb服务是启动的，但执行/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 会一直卡住。 求问后来如何解决的问题~

<!-- gh-comment-id:2818006578 --> @gongweicher commented on GitHub (Apr 21, 2025): > 我也发现这个一直卡住是什么原因，我foundationdb服务是启动的 /opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" hello, 我遇到了同样的问题。foundationdb服务是启动的，但执行/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 会一直卡住。 求问后来如何解决的问题~
bsakaguchi commented 2026-10-26 09:44:05 +00:00
Author
Owner
Copy link

@zhulin5211314 commented on GitHub (Apr 25, 2025):

我也发现这个一直卡住是什么原因，我foundationdb服务是启动的 /opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16"

hello, 我遇到了同样的问题。foundationdb服务是启动的，但执行/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 会一直卡住。 求问后来如何解决的问题~

加一下fdb的日志，看下是不是版本有问题

<!-- gh-comment-id:2829187819 --> @zhulin5211314 commented on GitHub (Apr 25, 2025): > > 我也发现这个一直卡住是什么原因，我foundationdb服务是启动的 /opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" > > hello, 我遇到了同样的问题。foundationdb服务是启动的，但执行/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 会一直卡住。 求问后来如何解决的问题~ 加一下fdb的日志，看下是不是版本有问题
bsakaguchi commented 2026-10-26 09:44:05 +00:00
Author
Owner
Copy link

@interestingyong commented on GitHub (Apr 27, 2025):

https://blog.open3fs.com/2025/03/28/deploy-3fs-with-m3fs.html
有人尝试过m3fs 直接部署3fs吗

<!-- gh-comment-id:2832905860 --> @interestingyong commented on GitHub (Apr 27, 2025): https://blog.open3fs.com/2025/03/28/deploy-3fs-with-m3fs.html 有人尝试过m3fs 直接部署3fs吗
bsakaguchi commented 2026-10-26 09:44:06 +00:00
Author
Owner
Copy link

@gongweicher commented on GitHub (Apr 28, 2025):

我也发现这个一直卡住是什么原因，我foundationdb服务是启动的 /opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16"

hello, 我遇到了同样的问题。foundationdb服务是启动的，但执行/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 会一直卡住。 求问后来如何解决的问题~

加一下fdb的日志，看下是不是版本有问题

我是两个容器内foundationdb客户端/服务端的libfdb_c.so版本没对齐 打平后就OK啦~ THX

<!-- gh-comment-id:2834441751 --> @gongweicher commented on GitHub (Apr 28, 2025): > > > 我也发现这个一直卡住是什么原因，我foundationdb服务是启动的 /opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" > > > > > > hello, 我遇到了同样的问题。foundationdb服务是启动的，但执行/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" 会一直卡住。 求问后来如何解决的问题~ > > 加一下fdb的日志，看下是不是版本有问题 我是两个容器内foundationdb客户端/服务端的libfdb_c.so版本没对齐 打平后就OK啦~ THX
bsakaguchi commented 2026-10-26 09:44:06 +00:00
Author
Owner
Copy link

@interestingyong commented on GitHub (Apr 28, 2025):

docker cp 3fs-fdb:/usr/lib/libfdb_c.so libfdb_c.so
Successfully copied 23.7MB to /root/libfdb_c.so

root@node220:~# docker cp 3fs-mgmtd:/usr/lib/libfdb_c.so libfdb_c.so_2
Successfully copied 23.7MB to /root/libfdb_c.so_2

root@node220:# md5sum libfdb_c.so
bc3b99a82c71b4ca254cc45ee042455a libfdb_c.so
root@node220:# md5sum libfdb_c.so_2
ab004522b444ef2bfffbfdefced5d834 libfdb_c.so_2

直接替换吗 @gongweicher

<!-- gh-comment-id:2834467072 --> @interestingyong commented on GitHub (Apr 28, 2025): docker cp 3fs-fdb:/usr/lib/libfdb_c.so libfdb_c.so Successfully copied 23.7MB to /root/libfdb_c.so root@node220:~# docker cp 3fs-mgmtd:/usr/lib/libfdb_c.so libfdb_c.so_2 Successfully copied 23.7MB to /root/libfdb_c.so_2 root@node220:~# md5sum libfdb_c.so bc3b99a82c71b4ca254cc45ee042455a libfdb_c.so root@node220:~# md5sum libfdb_c.so_2 ab004522b444ef2bfffbfdefced5d834 libfdb_c.so_2 直接替换吗 @gongweicher
bsakaguchi commented 2026-10-26 09:44:06 +00:00
Author
Owner
Copy link

@interestingyong commented on GitHub (Apr 28, 2025):

我用docker cp替换了文件 然后restart 还是不行
[2025-04-28T08:41:50.134543622+00:00 Collector: 9 Recorder.cc:98 DEBUG] Last 30s fdb_total_count_get: 1
[2025-04-28T08:44:45.474362672+00:00 admin_cli: 1 SysResource.cc:55 INFO] rlimit for number of files has already been met. 524288 <= 1048576
[2025-04-28T08:44:45.479609696+00:00 admin_cli: 1 admin_cli.cc:210 INFO] cli start cmd = "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" verbose = true profile = false
[2025-04-28T08:44:45.486021728+00:00 admin_cli: 1 MetaStore.cc:32 INFO] MetaStore::initFs
[2025-04-28T08:45:15.477343466+00:00 Collector: 9 Recorder.cc:98 DEBUG] Last 30s fdb_total_count_get: 1
[2025-04-28T08:45:29.512653746+00:00 admin_cli: 1 SysResource.cc:55 INFO] rlimit for number of files has already been met. 524288 <= 1048576
[2025-04-28T08:45:29.517859276+00:00 admin_cli: 1 admin_cli.cc:210 INFO] cli start cmd = "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" verbose = true profile = false
[2025-04-28T08:45:29.524296857+00:00 admin_cli: 1 MetaStore.cc:32 INFO] MetaStore::initFs
[2025-04-28T08:45:59.515701798+00:00 Collector: 8 Recorder.cc:98 DEBUG] Last 30s fdb_total_count_get: 1

<!-- gh-comment-id:2834479520 --> @interestingyong commented on GitHub (Apr 28, 2025): 我用docker cp替换了文件 然后restart 还是不行 [2025-04-28T08:41:50.134543622+00:00 Collector: 9 Recorder.cc:98 DEBUG] Last 30s fdb_total_count_get: 1 [2025-04-28T08:44:45.474362672+00:00 admin_cli: 1 SysResource.cc:55 INFO] rlimit for number of files has already been met. 524288 <= 1048576 [2025-04-28T08:44:45.479609696+00:00 admin_cli: 1 admin_cli.cc:210 INFO] cli start cmd = "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" verbose = true profile = false [2025-04-28T08:44:45.486021728+00:00 admin_cli: 1 MetaStore.cc:32 INFO] MetaStore::initFs [2025-04-28T08:45:15.477343466+00:00 Collector: 9 Recorder.cc:98 DEBUG] Last 30s fdb_total_count_get: 1 [2025-04-28T08:45:29.512653746+00:00 admin_cli: 1 SysResource.cc:55 INFO] rlimit for number of files has already been met. 524288 <= 1048576 [2025-04-28T08:45:29.517859276+00:00 admin_cli: 1 admin_cli.cc:210 INFO] cli start cmd = "init-cluster --mgmtd /opt/3fs/etc/mgmtd_main.toml 1 1048576 16" verbose = true profile = false [2025-04-28T08:45:29.524296857+00:00 admin_cli: 1 MetaStore.cc:32 INFO] MetaStore::initFs [2025-04-28T08:45:59.515701798+00:00 Collector: 8 Recorder.cc:98 DEBUG] Last 30s fdb_total_count_get: 1
bsakaguchi commented 2026-10-26 09:44:07 +00:00
Author
Owner
Copy link

@Hansyangxingxiang commented on GitHub (May 16, 2025):

build容器镜像以及容器化部署指导：
https://github.com/deepseek-ai/3FS/pull/278/files#diff-ca7e13c458ccb7bf066db5e364787a8420bc72b672d43e5bc881aaeef34b7110

<!-- gh-comment-id:2886038313 --> @Hansyangxingxiang commented on GitHub (May 16, 2025): build容器镜像以及容器化部署指导： https://github.com/deepseek-ai/3FS/pull/278/files#diff-ca7e13c458ccb7bf066db5e364787a8420bc72b672d43e5bc881aaeef34b7110
bsakaguchi commented 2026-10-26 09:44:07 +00:00
Author
Owner
Copy link

@interestingyong commented on GitHub (May 16, 2025):

build容器镜像以及容器化部署指导： https://github.com/deepseek-ai/3FS/pull/278/files#diff-ca7e13c458ccb7bf066db5e364787a8420bc72b672d43e5bc881aaeef34b7110

好人一生平安 我后续试试

<!-- gh-comment-id:2887113292 --> @interestingyong commented on GitHub (May 16, 2025): > build容器镜像以及容器化部署指导： https://github.com/deepseek-ai/3FS/pull/278/files#diff-ca7e13c458ccb7bf066db5e364787a8420bc72b672d43e5bc881aaeef34b7110 好人一生平安 我后续试试
bsakaguchi commented 2026-10-26 09:44:07 +00:00
Author
Owner
Copy link

@huww98 commented on GitHub (May 16, 2025):

Maybe it's time to package it as daemonset and helm charts for Kubernetes.

Take a look at our approach with helm chart

<!-- gh-comment-id:2887455421 --> @huww98 commented on GitHub (May 16, 2025): > Maybe it's time to package it as daemonset and helm charts for Kubernetes. Take a look at [our approach](https://github.com/AliyunContainerService/alibabacloud-3FS/blob/main/deploy/container/README.md) with [helm chart](https://github.com/AliyunContainerService/alibabacloud-3FS/tree/main/deploy/container/chart)
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
SF-Zhou-patch-1
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/3FS#43
No description provided.