deepseek-ai/3FS
1
0
Fork 0
mirror of https://github.com/deepseek-ai/3FS.git synced 2026-11-05 00:47:47 +00:00
Code Issues 166 Projects Packages Wiki Activity

[GH-ISSUE #296] got signal=7 if fio job_num is large #180

New issue
Closed
opened 2026-10-26 09:46:39 +00:00 by bsakaguchi · 1 comment
bsakaguchi commented 2026-10-26 09:46:39 +00:00
Owner
Copy link

Originally created by @lcxywfe on GitHub (Jun 13, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/296

root@029c49d66f62:~/fio# fio -numjobs=128 -fallocate=none
-ioengine=external:/usr/lib/hf3fs_usrbio.so
-direct=1
-rw=write
-bs=8MB
--group_reporting
-filesize=500MB
--nrfiles=10
-name=/3fs/test/test
-mountpoint=/3fs
-ior_depth=1
-iodepth=1

/3fs/test/test: Laying out IO files (100 files / total 50000MiB)
/3fs/test/test: Laying out IO files (100 files / total 50000MiB)
/3fs/test/test: Laying out IO files (100 files / total 50000MiB)
/3fs/test/test: Laying out IO files (100 files / total 50000MiB)
/3fs/test/test: Laying out IO files (100 files / total 50000MiB)
/3fs/test/test: Laying out IO files (100 files / total 50000MiB)
fio: pid=4888, got signal=7
fio: pid=4908, got signal=7
fio: pid=4913, got signal=7
fio: pid=4916, got signal=7
fio: pid=4922, got signal=7

and /var/log/3fs/hf3fs_fuse_main-err.log

[2025-06-22T16:38:50.734751409+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.734773285+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.735725208+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.735743384+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.736567686+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.736589394+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.737624447+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.737638620+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.739269779+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.739292095+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.740352844+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.740374240+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.741405746+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e52fc9000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.741425075+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e52fc9000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.742340692+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.742360203+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.743533308+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.743555190+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.744174592+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.744189563+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
[2025-06-22T16:38:50.745231908+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14
[2025-06-22T16:38:50.745252486+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg

Originally created by @lcxywfe on GitHub (Jun 13, 2025). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/296 > root@029c49d66f62:~/fio# fio -numjobs=128 -fallocate=none \ -ioengine=external:/usr/lib/hf3fs_usrbio.so \ -direct=1 \ -rw=write \ -bs=8MB \ --group_reporting \ -filesize=500MB \ --nrfiles=10 \ -name=/3fs/test/test \ -mountpoint=/3fs \ -ior_depth=1 \ -iodepth=1 /3fs/test/test: Laying out IO files (100 files / total 50000MiB) /3fs/test/test: Laying out IO files (100 files / total 50000MiB) /3fs/test/test: Laying out IO files (100 files / total 50000MiB) /3fs/test/test: Laying out IO files (100 files / total 50000MiB) /3fs/test/test: Laying out IO files (100 files / total 50000MiB) /3fs/test/test: Laying out IO files (100 files / total 50000MiB) fio: pid=4888, got signal=7 fio: pid=4908, got signal=7 fio: pid=4913, got signal=7 fio: pid=4916, got signal=7 fio: pid=4922, got signal=7 and /var/log/3fs/hf3fs_fuse_main-err.log [2025-06-22T16:38:50.734751409+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.734773285+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.735725208+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.735743384+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.736567686+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.736589394+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.737624447+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.737638620+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.739269779+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.739292095+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.740352844+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.740374240+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.741405746+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e52fc9000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.741425075+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e52fc9000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.742340692+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.742360203+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.743533308+00:00 CliBG0: 79 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.743555190+00:00 CliBG0: 79 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.744174592+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.744189563+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg [2025-06-22T16:38:50.745231908+00:00 CliBG1: 80 IBDevice.cc:515 CRITICAL] IBDevice mlx5_0 failed to reg_mr, addr 0x7f5e537cb000, length 8396798, access 1048583, errno 14 [2025-06-22T16:38:50.745252486+00:00 CliBG1: 80 Shm.cc:112 ERROR] failed to register buffer @0x7f5e537cb000 seg #0 with bytes 8396798 block size 8396798 code 7001 msg
bsakaguchi commented 2026-10-26 09:46:41 +00:00
Author
Owner
Copy link

@wwj-2017-1117 commented on GitHub (Dec 4, 2025):

me too

<!-- gh-comment-id:3613722335 --> @wwj-2017-1117 commented on GitHub (Dec 4, 2025): me too
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
SF-Zhou-patch-1
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/3FS#180
No description provided.