shm not deleted #203

New issue

Open

opened 2026-10-26 09:47:05 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:47:05 +00:00

Owner

Originally created by @McFurryIsNotMcFlurry on GitHub (Aug 31, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/334

When a client process reading files via hf3fs_fuse.io APIs exited abruptly (such as being killed by SIGKILL), the symlinks under /3fs/3fs-virt/iovs/ are not removed (because iovec.__del__ is not called), and since those symlinks are pointing to shared memories under /dev/shm, those shared memories will never be freed unless someone manually delete those symlinks, or restart the entire hf3fs_fuse_main service.

And of course there're other scenarios in which iovec.__del__ is not called, for example malformed user programs which contains cyclic reference. But if I expect 3FS to serve as a robust fundamental filesystem, which should withstand improper user use cases, what is the proper way to cope with this memory leakage issue?

Originally created by @McFurryIsNotMcFlurry on GitHub (Aug 31, 2025). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/334 When a client process reading files via `hf3fs_fuse.io` APIs exited abruptly (such as being killed by SIGKILL), the symlinks under `/3fs/3fs-virt/iovs/` are not removed (because `iovec.__del__` is not called), and since those symlinks are pointing to shared memories under `/dev/shm`, those shared memories will never be freed unless someone manually delete those symlinks, or restart the entire `hf3fs_fuse_main` service. And of course there're other scenarios in which `iovec.__del__` is not called, for example malformed user programs which contains cyclic reference. But if I expect 3FS to serve as a robust fundamental filesystem, which should withstand improper user use cases, what is the proper way to cope with this memory leakage issue?

bsakaguchi commented

2026-10-26 09:47:06 +00:00

Author

Owner

@McFurryIsNotMcFlurry commented on GitHub (Sep 1, 2025):

Further investigation shows that, after read process termination, those symlinks under /3fs/3fs-virt/iovs/ are actually dangling - they're pointing to non-existent items under /dev/shm. I believe those items under /dev/shm, created by multiprocessing.shared_memory.SharedMemory, are cleared by multiprocessing.resource_tracker, a daemon that does the clean-up job after the read process exited abruptly. However the iovec objects lack such mechanism, and thus those links under /3fs/3fs-virt/iovs/ will remain after abrupt exit.

This raises more question - why do those dangling symlinks under /3fs/3fs-virt/iovs/ affect the amount of residual shared memory, since manually delete those symlinks will actually cause the residual memory of hf3fs_fuse_main process to drop?

@McFurryIsNotMcFlurry commented on GitHub (Sep 1, 2025): Further investigation shows that, after read process termination, those symlinks under `/3fs/3fs-virt/iovs/` are actually dangling - they're pointing to non-existent items under `/dev/shm`. I believe those items under `/dev/shm`, created by `multiprocessing.shared_memory.SharedMemory`, are cleared by `multiprocessing.resource_tracker`, a daemon that does the clean-up job after the read process exited abruptly. However the `iovec` objects lack such mechanism, and thus those links under `/3fs/3fs-virt/iovs/` will remain after abrupt exit. This raises more question - why do those dangling symlinks under `/3fs/3fs-virt/iovs/` affect the amount of residual shared memory, since manually delete those symlinks will actually cause the residual memory of `hf3fs_fuse_main` process to drop?

bsakaguchi commented

2026-10-26 09:47:06 +00:00

Author

Owner

@lcxywfe commented on GitHub (Jan 8, 2026):

Same issue

@lcxywfe commented on GitHub (Jan 8, 2026): Same issue

bsakaguchi referenced this issue

2026-10-26 09:48:42 +00:00

[PR #203] [CLOSED] delete unnecessary debug code #326