[GH-ISSUE #136] Seeking Clarification on Data Transfer, KVCache Reading, and Optimization Routes #78

New issue

Open

opened 2026-10-26 09:44:39 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 09:44:39 +00:00

Owner

Originally created by @unbelievable3513 on GitHub (Mar 10, 2025).
Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/136

I apologize for raising an issue here to seek clarification, as despite conducting extensive research, I'm still confused. From my understanding, in the context of data transfer from the client to the server in 3FS, the path is
client_DRAM -> RDMA -> server_DRAM -> directIO -> write to SSD.
Is this correct?

Regarding reading the KVCache via 3FS, is the transfer route as follows:
server_SSD -> directIO -> server_DRAM -> RDMA -> client_DRAM -> memcpy to GPU memory?

Also, considering the availability of technologies like GPU Direct RDMA or GPU direct storage (such as with the A100), is it possible to optimize the route to
server_SSD -> RDMA -> GPU_memory?

Or could you please tell me where I can obtain further materials and plans regarding these topics.

Thank you in advance for your time. I look forward to your insights.

Originally created by @unbelievable3513 on GitHub (Mar 10, 2025). Original GitHub issue: https://github.com/deepseek-ai/3FS/issues/136 I apologize for raising an issue here to seek clarification, as despite conducting extensive research, I'm still confused. From my understanding, in the context of data transfer from the client to the server in 3FS, the path is ```client_DRAM -> RDMA -> server_DRAM -> directIO -> write to SSD.``` Is this correct? Regarding reading the KVCache via 3FS, is the transfer route as follows: ```server_SSD -> directIO -> server_DRAM -> RDMA -> client_DRAM -> memcpy to GPU memory```? Also, considering the availability of technologies like GPU Direct RDMA or GPU direct storage (such as with the A100), is it possible to optimize the route to ```server_SSD -> RDMA -> GPU_memory```? Or could you please tell me where I can obtain further materials and plans regarding these topics. Thank you in advance for your time. I look forward to your insights.

bsakaguchi commented

2026-10-26 09:44:40 +00:00

Author

Owner

@wangyibin-gh commented on GitHub (Mar 13, 2025):

AFAICS, gds isn't implemented in 3fs code. kvcache should be managed using seperated software, which isn't opensourced, yet. I think 3fs-kv isn't opensourced just because it is tightly coupled with inference engine and this makes 3fs-kv less powerful without the logic in the inference engine.

@wangyibin-gh commented on GitHub (Mar 13, 2025): AFAICS, gds isn't implemented in 3fs code. kvcache should be managed using seperated software, which isn't opensourced, yet. I think 3fs-kv isn't opensourced just because it is tightly coupled with inference engine and this makes 3fs-kv less powerful without the logic in the inference engine.

bsakaguchi referenced this issue

2026-10-26 09:48:14 +00:00

[PR #80] [CLOSED] docs: build with docker (#78). #282

bsakaguchi referenced this issue

2026-10-26 09:48:15 +00:00

[PR #81] [CLOSED] docs: build with docker (#78). #283