[GH-ISSUE #428] Opt-in rusty_alloc backend for MimallocLib #245

New issue

Open

opened 2026-10-26 09:47:43 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 09:47:43 +00:00

Owner

Originally created by @Ttimmahlax on GitHub (Sep 4, 2026).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/428

The value of integrating

3FS is currently pulling in mimalloc for allocation; src/memory/mimalloc/MimallocLib.cc includes mimalloc.h.

If you added rusty_alloc as an opt-in backend, the same mi_* surface would run on a ground-up Rust allocator. A double free aborts on both the owner and cross-thread paths (mimalloc in release does not). USRBIO, CRAQ and the file interface stay as they are. The rusty_alloc README measures lua 0.97×, perl 0.99× and sqlite 1.00× mimalloc's whole-program instructions retired (LD_PRELOAD).

Not asking you to change the default; this would sit behind a compile-time flag.

How to integrate

You already sit mimalloc behind MemoryAllocatorInterface. Add rusty_alloc_ffi as an off-by-default backend. It is the mi_*-compatible C ABI in the rusty_alloc 1.1.6 tree. Today's mimalloc path stays the default.

rusty_alloc = "1.1.6"

Call sites stay mi_*. The names do not change. See the rusty_alloc README (Architecture) for the FFI crate. Builders need rustc for that crate; nothing else in the 3FS tree moves. MemoryAllocatorInterface, the file API, tests and on-disk format stay as they are.

Turn it on in CI, compare on your own corpus, and drop it with one line if it does not hold up.

Remade With Rust & MATA

Remade With Rust is rebuilding the stack in safe Rust — codecs, parsers, compression, identity, storage — so projects can drop C dependencies without giving up performance. If anything else in the org fits the 3FS build, it lives at https://github.com/Remade-With-Rust. Close this issue if it is a poor fit.

x.com/farmer_timmm

Originally created by @Ttimmahlax on GitHub (Sep 4, 2026). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/428 ### The value of integrating 3FS is currently pulling in mimalloc for allocation; `src/memory/mimalloc/MimallocLib.cc` includes `mimalloc.h`. If you added rusty_alloc as an opt-in backend, the same `mi_*` surface would run on a ground-up Rust allocator. A double free aborts on both the owner and cross-thread paths (mimalloc in release does not). USRBIO, CRAQ and the file interface stay as they are. The rusty_alloc README measures lua 0.97×, perl 0.99× and sqlite 1.00× mimalloc's whole-program instructions retired (`LD_PRELOAD`). Not asking you to change the default; this would sit behind a compile-time flag. ### How to integrate You already sit mimalloc behind `MemoryAllocatorInterface`. Add `rusty_alloc_ffi` as an off-by-default backend. It is the `mi_*`-compatible C ABI in the rusty_alloc 1.1.6 tree. Today's mimalloc path stays the default. ```toml rusty_alloc = "1.1.6" ``` Call sites stay `mi_*`. The names do not change. See the rusty_alloc README (Architecture) for the FFI crate. Builders need rustc for that crate; nothing else in the 3FS tree moves. `MemoryAllocatorInterface`, the file API, tests and on-disk format stay as they are. Turn it on in CI, compare on your own corpus, and drop it with one line if it does not hold up. ### Remade With Rust & MATA Remade With Rust is rebuilding the stack in safe Rust — codecs, parsers, compression, identity, storage — so projects can drop C dependencies without giving up performance. If anything else in the org fits the 3FS build, it lives at https://github.com/Remade-With-Rust. Close this issue if it is a poor fit. - x.com/farmer_timmm

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/3FS#245

No description provided.

Rows
Columns