mirror of
https://github.com/deepseek-ai/3FS.git
synced 2026-11-05 00:47:47 +00:00
[GH-ISSUE #409] any plan to support erasure coding? #234
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/3FS#234
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @hjk41 on GitHub (Apr 9, 2026).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/409
We would love to place all data on 3fs, but code data is taking too much space. It would be great if 3fs can support erasure coding on specified directories.
@hiSandog commented on GitHub (Jun 27, 2026):
Per-directory erasure coding would be useful, but the policy boundary is probably the hard part. Would the intended model be a directory-level storage class inherited by new files, or a migration job that can rewrite existing replicated data into EC chunks? For AI training data, it would also help to define the expected read-repair and degraded-read behavior, since latency spikes during reconstruction could matter more than raw capacity savings.
@hjk41 commented on GitHub (Jun 28, 2026):
Directory level storage class would be great. Currently we are storing cold data in a Ceph cluster and copying training data to 3fs when needed. If 3fs supports erasure coding, we can simply store everyting in one single 3fs with cold data in erasure-coding directory and training data in a replication directory, which would save the effort of maintaining two different systems.