[GH-ISSUE #150] when the system restart, the mgmtd_main coredum, as well the server can not start up again #87

Open
opened 2026-10-26 09:44:51 +00:00 by bsakaguchi · 1 comment
Owner

Originally created by @xiaoyaoyouyue on GitHub (Mar 11, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/150

root@ubuntu:~# systemctl status mgmtd_main
× mgmtd_main.service - mgmtd_main Server
Loaded: loaded (/lib/systemd/system/mgmtd_main.service; disabled; vendor preset: enabled)
Active: failed (Result: signal) since Tue 2025-03-11 16:58:37 CST; 3min 21s ago
Process: 4893 ExecStart=/opt/3fs/bin/mgmtd_main --launcher_cfg /opt/3fs/etc/mgmtd_main_launcher.toml --app-cfg /opt/3fs/etc/mgmtd_main_app.toml (code=killed, signal=ABRT)
Main PID: 4893 (code=killed, signal=ABRT)
CPU: 489ms

Mar 11 16:58:37 ubuntu mgmtd_main[4893]: -> /root/3fs/3fs/src/mgmtd/mgmtd.cpp
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000004ccf96 _ZN5hf3fs15ApplicationBase3runEiPPc
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/common/app/ApplicationBase.cc:65
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000932f2d main
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/mgmtd/mgmtd.cpp:7
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000029d8f (unknown)
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000029e3f __libc_start_main
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 000000000022a0a4 _start
Mar 11 16:58:37 ubuntu systemd[1]: mgmtd_main.service: Main process exited, code=killed, status=6/ABRT
Mar 11 16:58:37 ubuntu systemd[1]: mgmtd_main.service: Failed with result 'signal'.

Mar 11 16:58:37 ubuntu mgmtd_main[4893]: [2025-03-11T16:58:37.216115350+08:00 mgmtd_main: 4893 Utils.cc:161 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/mgmtd_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/mgmtd_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}}
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: [2025-03-11T16:58:37.505520811+08:00 mgmtd_main: 4893 TwoPhaseApplication.h:59 FATAL] Init server failed: RPC::ListenFailed(2011)
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: *** Aborted at 1741683517 (Unix time, try 'date -d @1741683517') ***
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: *** Signal 6 (SIGABRT) (0x131d) received by PID 4893 (pthread TID 0x7f717288e600) (linux TID 4893) (maybe from PID 4893, UID 0) (code: -6), stack trace: ***
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000750e8f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 000000000004251f (unknown)
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000000969fc pthread_kill
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000042475 raise
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000000287f2 abort
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000007b8b50 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/third_party/folly/folly/logging/LogCategory.cpp:71
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000007b8006 _ZN5folly18LogStreamProcessor6logNowEv
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000007b8259 _ZN5folly16LogStreamVoidifyILb1EEanERSo
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000935ee3 _ZN5hf3fs19TwoPhaseApplicationINS_5mgmtd11MgmtdServerEE15initApplicationEv
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/common/app/TwoPhaseApplication.h:59
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: -> /root/3fs/3fs/src/mgmtd/mgmtd.cpp
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000004ccf96 _ZN5hf3fs15ApplicationBase3runEiPPc
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/common/app/ApplicationBase.cc:65
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000932f2d main
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/mgmtd/mgmtd.cpp:7
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000029d8f (unknown)
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000029e3f __libc_start_main
Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 000000000022a0a4 _start
Mar 11 16:58:37 ubuntu systemd[1]: mgmtd_main.service: Main process exited, code=killed, status=6/ABRT
Mar 11 16:58:37 ubuntu systemd[1]: mgmtd_main.service: Failed with result 'signal'.

Originally created by @xiaoyaoyouyue on GitHub (Mar 11, 2025). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/150 root@ubuntu:~# systemctl status mgmtd_main × mgmtd_main.service - mgmtd_main Server Loaded: loaded (/lib/systemd/system/mgmtd_main.service; disabled; vendor preset: enabled) Active: failed (Result: signal) since Tue 2025-03-11 16:58:37 CST; 3min 21s ago Process: 4893 ExecStart=/opt/3fs/bin/mgmtd_main --launcher_cfg /opt/3fs/etc/mgmtd_main_launcher.toml --app-cfg /opt/3fs/etc/mgmtd_main_app.toml (code=killed, signal=ABRT) Main PID: 4893 (code=killed, signal=ABRT) CPU: 489ms Mar 11 16:58:37 ubuntu mgmtd_main[4893]: -> /root/3fs/3fs/src/mgmtd/mgmtd.cpp Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000004ccf96 _ZN5hf3fs15ApplicationBase3runEiPPc Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/common/app/ApplicationBase.cc:65 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000932f2d main Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/mgmtd/mgmtd.cpp:7 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000029d8f (unknown) Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000029e3f __libc_start_main Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 000000000022a0a4 _start Mar 11 16:58:37 ubuntu systemd[1]: mgmtd_main.service: Main process exited, code=killed, status=6/ABRT Mar 11 16:58:37 ubuntu systemd[1]: mgmtd_main.service: Failed with result 'signal'. Mar 11 16:58:37 ubuntu mgmtd_main[4893]: [2025-03-11T16:58:37.216115350+08:00 mgmtd_main: 4893 Utils.cc:161 INFO] LogConfig: {"categories":{".":{"level":"INFO","inherit":true,"propagate":"NONE","handlers":["normal","err","fatal"]}},"handlers":{"normal":{"type":"file","options":{"path":"/var/log/3fs/mgmtd_main.log","async":"true","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"err":{"type":"file","options":{"level":"ERR","path":"/var/log/3fs/mgmtd_main-err.log","async":"false","rotate":"true","max_files":"10","max_file_size":"104857600","rotate_on_open":"false"}},"fatal":{"type":"stream","options":{"level":"FATAL","stream":"stderr"}}}} Mar 11 16:58:37 ubuntu mgmtd_main[4893]: [2025-03-11T16:58:37.505520811+08:00 mgmtd_main: 4893 TwoPhaseApplication.h:59 FATAL] Init server failed: RPC::ListenFailed(2011) Mar 11 16:58:37 ubuntu mgmtd_main[4893]: *** Aborted at 1741683517 (Unix time, try 'date -d @1741683517') *** Mar 11 16:58:37 ubuntu mgmtd_main[4893]: *** Signal 6 (SIGABRT) (0x131d) received by PID 4893 (pthread TID 0x7f717288e600) (linux TID 4893) (maybe from PID 4893, UID 0) (code: -6), stack trace: *** Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000750e8f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 000000000004251f (unknown) Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000000969fc pthread_kill Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000042475 raise Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000000287f2 abort Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000007b8b50 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/third_party/folly/folly/logging/LogCategory.cpp:71 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000007b8006 _ZN5folly18LogStreamProcessor6logNowEv Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000007b8259 _ZN5folly16LogStreamVoidifyILb1EEanERSo Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000935ee3 _ZN5hf3fs19TwoPhaseApplicationINS_5mgmtd11MgmtdServerEE15initApplicationEv Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/common/app/TwoPhaseApplication.h:59 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: -> /root/3fs/3fs/src/mgmtd/mgmtd.cpp Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 00000000004ccf96 _ZN5hf3fs15ApplicationBase3runEiPPc Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/common/app/ApplicationBase.cc:65 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000932f2d main Mar 11 16:58:37 ubuntu mgmtd_main[4893]: /root/3fs/3fs/src/mgmtd/mgmtd.cpp:7 Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000029d8f (unknown) Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 0000000000029e3f __libc_start_main Mar 11 16:58:37 ubuntu mgmtd_main[4893]: @ 000000000022a0a4 _start Mar 11 16:58:37 ubuntu systemd[1]: mgmtd_main.service: Main process exited, code=killed, status=6/ABRT Mar 11 16:58:37 ubuntu systemd[1]: mgmtd_main.service: Failed with result 'signal'.
Author
Owner

@Icedroid commented on GitHub (Mar 15, 2025):

我也遇到一模一样的问题,大佬是怎么解决的?

<!-- gh-comment-id:2726188121 --> @Icedroid commented on GitHub (Mar 15, 2025): 我也遇到一模一样的问题,大佬是怎么解决的?
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/3FS#87
No description provided.