[GH-ISSUE #274] 海光四机器部署3FS的FUSE,admin_cli 报core dump错误。 #167

Open
opened 2026-10-26 09:46:30 +00:00 by bsakaguchi · 1 comment
Owner

Originally created by @wjluo on GitHub (May 14, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/274

环境:
1、海光四型号为Hygon C86-4G (OPN:7490)。Byte Order为 Little Endian
2、操作系统为Ubuntu 22.04.5 LTS。
3、kernel版本为5.15.0-136-generic

4、出错信息如下:
/usr/local/3fs/bin/admin_cli -cfg /usr/local/3fs/conf/admin_cli.toml --config.mgmtd_client.mgmtd_server_addresses '["RDMA://192.168.15.81:8000"]' "set-config --type FUSE --file /usr/local/3fs/conf/hf3fs_fuse_main.toml"
*** Aborted at 1747229966 (Unix time, try 'date -d @1747229966') ***
*** Signal 6 (SIGABRT) (0x1e5648) received by PID 1988168 (pthread TID 0x7fa4d962d700) (linux TID 1988168) (maybe from PID 1988168, UID 0) (code: -6), stack trace: ***
@ 00000000020c526f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv
/workspace/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449
@ 000000000004251f (unknown)
@ 00000000000969fc pthread_kill
@ 0000000000042475 raise
@ 00000000000287f2 abort
@ 000000000212c810 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE
/workspace/3fs/third_party/folly/folly/logging/LogCategory.cpp:71
@ 000000000212bcc6 _ZN5folly18LogStreamProcessor6logNowEv
/workspace/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190
@ 000000000212bf19 _ZN5folly16LogStreamVoidifyILb1EEanERSo
/workspace/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222
@ 00000000015a2613 _ZZZ4mainENK4$_12clEvENKUlvE_clEv
/workspace/3fs/src/client/bin/admin_cli.cc:107
@ 00000000015a244b _ZZZ4mainENK4$_18clEvENKUlvE_clEv
/workspace/3fs/src/client/bin/admin_cli.cc:105
@ 00000000015a2c2d _ZZZ4mainENK4$_19clEvENKUlvE_clEv
/workspace/3fs/src/client/bin/admin_cli.cc:118
@ 00000000015a27f1 _ZNSt17_Function_handlerIFSt10shared_ptrIN5hf3fs6client20IMgmtdClientForAdminEEvEZ4mainE3$_3E9_M_invokeERKSt9_Any_data
/workspace/3fs/src/client/bin/admin_cli.cc:127
@ 0000000001935517 _ZN5hf3fs6client3cli12_GLOBAL__N_16handleERNS1_4IEnvERKN8argparse14ArgumentParserERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISF_EE.resume
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:591
-> /workspace/3fs/src/client/cli/admin/SetConfig.cc
@ 0000000002150723 _ZN5folly36resumeCoroutineWithNewAsyncStackRootENSt7__n486116coroutine_handleIvEERNS_15AsyncStackFrameE
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/coroutine:135
-> /workspace/3fs/third_party/folly/folly/tracing/AsyncStack.cpp
@ 000000000165cb96 _ZN5folly6detail8function14FunctionTraitsIFvvEE9callSmallIZNOS_4coro6detail16BlockingWaitTaskIRNS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEE6getViaEPNS_16DrivableExecutorERNS_15AsyncStackFrameEEUlvE_EEvRNS1_4DataE
/workspace/3fs/third_party/folly/folly/tracing/AsyncStack-inl.h:88
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 000000000165be1d _ZZN5folly4coro6detail20BlockingWaitExecutor5driveEvENKUlvE_clEv
/workspace/3fs/third_party/folly/folly/Function.h:375
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 000000000165b9c3 _ZN5folly4coro6detail20BlockingWaitExecutor5driveEv
/workspace/3fs/third_party/folly/folly/fibers/FiberManagerInternal-inl.h:639
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 000000000165c9c2 _ZNO5folly4coro6detail16BlockingWaitTaskIRNS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEE6getViaEPNS_16DrivableExecutorERNS_15AsyncStackFrameE
/workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:235
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 000000000165c83d _ZNK5folly4coro16blocking_wait_fnclINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEEEEENS0_6detail22decay_rvalue_referenceINS0_12await_resultIDTclL_ZNS0_11folly_cpo__13co_viaIfAsyncEEclL_ZSt7declvalINS_8Executor9KeepAliveISF_EEEDTcl9__declvalIT_ELi0EEEvEEclsr3stdE7declvalISI_EEEEvE4typeEE4typeEOSI_PNS_16DrivableExecutorE
/workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:415
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 00000000015a6435 ZNK5folly4coro16blocking_wait_fnclINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEEELi0EEENS0_6detail22decay_rvalue_referenceINS0_12await_resultIDTclL_ZNS0_11folly_cpo__13co_viaIfAsyncEEclL_ZSt7declvalINS_8Executor9KeepAliveISF_EEEDTcl9__declvalIT_ELi0EEEvEEclsr3stdE7declvalISI_EEEEvE4typeEE4typeEOSI
/workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:427
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 00000000015961df main
/workspace/3fs/src/client/bin/admin_cli.cc:214
@ 0000000000029d8f (unknown)
@ 0000000000029e3f __libc_start_main
@ 0000000001594ea4 _start

*** Check failure async stack trace: ***
*** First async stack root: 0x7fffb7444858, normal stack frame pointer holding async stack root: 0x7fffb7444890, return address: 0x5644b5112b97 ***
*** First async stack frame pointer: 0x5644db262028, return address: 0x5644b568e8e2, async stack trace: ***
@ 0000000001bd88e2 _ZNK5hf3fs6client3cli10Dispatcher3runERNS1_4IEnvERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISB_EE.resume
/workspace/3fs/src/client/cli/common/Dispatcher.cc:0
@ 0000000001bd9654 _ZN5hf3fs6client3cli12_GLOBAL__N_17runLineB5cxx11ERNS1_4IEnvERNS1_10DispatcherESt17basic_string_viewIcSt11char_traitsIcEE.resume
/workspace/3fs/src/client/cli/common/Dispatcher.cc:0
@ 0000000001bda876 _ZN5hf3fs6client3cli10Dispatcher3runERNS1_4IEnvERKSt8functionIFNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEvEESt17basic_string_viewIcS9_Ebbb.resume
/workspace/3fs/src/client/cli/common/Dispatcher.cc:0
@ 00000000015a5cc5 _ZZ4mainENK3$_8clEv.resume
/workspace/3fs/src/client/bin/admin_cli.cc:0
@ 00000000015a559d ZN5folly4coro6detail23makeRefBlockingWaitTaskINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEE7AwaiterES8_Li0EEENS1_16BlockingWaitTaskINSt20add_lvalue_referenceIT0_E4typeEEEOT.resume
/workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:296
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 000000000165c83e _ZNK5folly4coro16blocking_wait_fnclINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEEEEENS0_6detail22decay_rvalue_referenceINS0_12await_resultIDTclL_ZNS0_11folly_cpo__13co_viaIfAsyncEEclL_ZSt7declvalINS_8Executor9KeepAliveISF_EEEDTcl9__declvalIT_ELi0EEEvEEclsr3stdE7declvalISI_EEEEvE4typeEE4typeEOSI_PNS_16DrivableExecutorE
/workspace/3fs/third_party/folly/folly/Expected.h:366
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 00000000015a6436 ZNK5folly4coro16blocking_wait_fnclINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEEELi0EEENS0_6detail22decay_rvalue_referenceINS0_12await_resultIDTclL_ZNS0_11folly_cpo__13co_viaIfAsyncEEclL_ZSt7declvalINS_8Executor9KeepAliveISF_EEEDTcl9__declvalIT_ELi0EEEvEEclsr3stdE7declvalISI_EEEEvE4typeEE4typeEOSI
/workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:301
-> /workspace/3fs/src/client/bin/admin_cli.cc
@ 00007fffb7444c80 (not found)
Aborted (core dumped)

Originally created by @wjluo on GitHub (May 14, 2025). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/274 环境: 1、海光四型号为Hygon C86-4G (OPN:7490)。Byte Order为 Little Endian 2、操作系统为Ubuntu 22.04.5 LTS。 3、kernel版本为5.15.0-136-generic 4、出错信息如下: /usr/local/3fs/bin/admin_cli -cfg /usr/local/3fs/conf/admin_cli.toml --config.mgmtd_client.mgmtd_server_addresses '["RDMA://192.168.15.81:8000"]' "set-config --type FUSE --file /usr/local/3fs/conf/hf3fs_fuse_main.toml" *** Aborted at 1747229966 (Unix time, try 'date -d @1747229966') *** *** Signal 6 (SIGABRT) (0x1e5648) received by PID 1988168 (pthread TID 0x7fa4d962d700) (linux TID 1988168) (maybe from PID 1988168, UID 0) (code: -6), stack trace: *** @ 00000000020c526f _ZN5folly10symbolizer12_GLOBAL__N_113signalHandlerEiP9siginfo_tPv /workspace/3fs/third_party/folly/folly/experimental/symbolizer/SignalHandler.cpp:449 @ 000000000004251f (unknown) @ 00000000000969fc pthread_kill @ 0000000000042475 raise @ 00000000000287f2 abort @ 000000000212c810 _ZNK5folly11LogCategory12admitMessageERKNS_10LogMessageE /workspace/3fs/third_party/folly/folly/logging/LogCategory.cpp:71 @ 000000000212bcc6 _ZN5folly18LogStreamProcessor6logNowEv /workspace/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:190 @ 000000000212bf19 _ZN5folly16LogStreamVoidifyILb1EEanERSo /workspace/3fs/third_party/folly/folly/logging/LogStreamProcessor.cpp:222 @ 00000000015a2613 _ZZZ4mainENK4$_12clEvENKUlvE_clEv /workspace/3fs/src/client/bin/admin_cli.cc:107 @ 00000000015a244b _ZZZ4mainENK4$_18clEvENKUlvE_clEv /workspace/3fs/src/client/bin/admin_cli.cc:105 @ 00000000015a2c2d _ZZZ4mainENK4$_19clEvENKUlvE_clEv /workspace/3fs/src/client/bin/admin_cli.cc:118 @ 00000000015a27f1 _ZNSt17_Function_handlerIFSt10shared_ptrIN5hf3fs6client20IMgmtdClientForAdminEEvEZ4mainE3$_3E9_M_invokeERKSt9_Any_data /workspace/3fs/src/client/bin/admin_cli.cc:127 @ 0000000001935517 _ZN5hf3fs6client3cli12_GLOBAL__N_16handleERNS1_4IEnvERKN8argparse14ArgumentParserERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISF_EE.resume /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:591 -> /workspace/3fs/src/client/cli/admin/SetConfig.cc @ 0000000002150723 _ZN5folly36resumeCoroutineWithNewAsyncStackRootENSt7__n486116coroutine_handleIvEERNS_15AsyncStackFrameE /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/coroutine:135 -> /workspace/3fs/third_party/folly/folly/tracing/AsyncStack.cpp @ 000000000165cb96 _ZN5folly6detail8function14FunctionTraitsIFvvEE9callSmallIZNOS_4coro6detail16BlockingWaitTaskIRNS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEE6getViaEPNS_16DrivableExecutorERNS_15AsyncStackFrameEEUlvE_EEvRNS1_4DataE /workspace/3fs/third_party/folly/folly/tracing/AsyncStack-inl.h:88 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 000000000165be1d _ZZN5folly4coro6detail20BlockingWaitExecutor5driveEvENKUlvE_clEv /workspace/3fs/third_party/folly/folly/Function.h:375 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 000000000165b9c3 _ZN5folly4coro6detail20BlockingWaitExecutor5driveEv /workspace/3fs/third_party/folly/folly/fibers/FiberManagerInternal-inl.h:639 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 000000000165c9c2 _ZNO5folly4coro6detail16BlockingWaitTaskIRNS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEE6getViaEPNS_16DrivableExecutorERNS_15AsyncStackFrameE /workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:235 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 000000000165c83d _ZNK5folly4coro16blocking_wait_fnclINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEEEEENS0_6detail22decay_rvalue_referenceINS0_12await_resultIDTclL_ZNS0_11folly_cpo__13co_viaIfAsyncEEclL_ZSt7declvalINS_8Executor9KeepAliveISF_EEEDTcl9__declvalIT_ELi0EEEvEEclsr3stdE7declvalISI_EEEEvE4typeEE4typeEOSI_PNS_16DrivableExecutorE /workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:415 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 00000000015a6435 _ZNK5folly4coro16blocking_wait_fnclINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEEELi0EEENS0_6detail22decay_rvalue_referenceINS0_12await_resultIDTclL_ZNS0_11folly_cpo__13co_viaIfAsyncEEclL_ZSt7declvalINS_8Executor9KeepAliveISF_EEEDTcl9__declvalIT_ELi0EEEvEEclsr3stdE7declvalISI_EEEEvE4typeEE4typeEOSI_ /workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:427 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 00000000015961df main /workspace/3fs/src/client/bin/admin_cli.cc:214 @ 0000000000029d8f (unknown) @ 0000000000029e3f __libc_start_main @ 0000000001594ea4 _start *** Check failure async stack trace: *** *** First async stack root: 0x7fffb7444858, normal stack frame pointer holding async stack root: 0x7fffb7444890, return address: 0x5644b5112b97 *** *** First async stack frame pointer: 0x5644db262028, return address: 0x5644b568e8e2, async stack trace: *** @ 0000000001bd88e2 _ZNK5hf3fs6client3cli10Dispatcher3runERNS1_4IEnvERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISB_EE.resume /workspace/3fs/src/client/cli/common/Dispatcher.cc:0 @ 0000000001bd9654 _ZN5hf3fs6client3cli12_GLOBAL__N_17runLineB5cxx11ERNS1_4IEnvERNS1_10DispatcherESt17basic_string_viewIcSt11char_traitsIcEE.resume /workspace/3fs/src/client/cli/common/Dispatcher.cc:0 @ 0000000001bda876 _ZN5hf3fs6client3cli10Dispatcher3runERNS1_4IEnvERKSt8functionIFNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEvEESt17basic_string_viewIcS9_Ebbb.resume /workspace/3fs/src/client/cli/common/Dispatcher.cc:0 @ 00000000015a5cc5 _ZZ4mainENK3$_8clEv.resume /workspace/3fs/src/client/bin/admin_cli.cc:0 @ 00000000015a559d _ZN5folly4coro6detail23makeRefBlockingWaitTaskINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEE7AwaiterES8_Li0EEENS1_16BlockingWaitTaskINSt20add_lvalue_referenceIT0_E4typeEEEOT_.resume /workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:296 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 000000000165c83e _ZNK5folly4coro16blocking_wait_fnclINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEEEEENS0_6detail22decay_rvalue_referenceINS0_12await_resultIDTclL_ZNS0_11folly_cpo__13co_viaIfAsyncEEclL_ZSt7declvalINS_8Executor9KeepAliveISF_EEEDTcl9__declvalIT_ELi0EEEvEEclsr3stdE7declvalISI_EEEEvE4typeEE4typeEOSI_PNS_16DrivableExecutorE /workspace/3fs/third_party/folly/folly/Expected.h:366 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 00000000015a6436 _ZNK5folly4coro16blocking_wait_fnclINS0_4TaskINS_8ExpectedINS_4UnitEN5hf3fs6StatusEEEEELi0EEENS0_6detail22decay_rvalue_referenceINS0_12await_resultIDTclL_ZNS0_11folly_cpo__13co_viaIfAsyncEEclL_ZSt7declvalINS_8Executor9KeepAliveISF_EEEDTcl9__declvalIT_ELi0EEEvEEclsr3stdE7declvalISI_EEEEvE4typeEE4typeEOSI_ /workspace/3fs/third_party/folly/folly/experimental/coro/BlockingWait.h:301 -> /workspace/3fs/src/client/bin/admin_cli.cc @ 00007fffb7444c80 (not found) Aborted (core dumped)
Author
Owner

@463975089 commented on GitHub (Dec 30, 2025):

请问问题解决了吗

<!-- gh-comment-id:3697980388 --> @463975089 commented on GitHub (Dec 30, 2025): 请问问题解决了吗
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/3FS#167
No description provided.