[GH-ISSUE #460] Hardware and Configuration Guidelines for DeepSeek-R1 "Full Version" Deployment #369

New issue

Closed

opened 2026-10-26 21:20:10 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-26 21:20:10 +00:00

Owner

Originally created by @Zephyrsailor on GitHub (Feb 20, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/460

Description:

We are planning to deploy the DeepSeek-R1 "full version" in a production environment and would like to confirm certain hardware and configuration requirements to ensure smooth deployment. Below are the key questions we have regarding the hardware configuration:

1. Model Precision:

FP8 Precision: Could you please confirm if the official "full version" uses FP8 precision?

2. Hardware Configuration:

GPU Configuration: For single-instance deployment, what are the recommended minimum and optimal GPU configurations? For example, is 8×A100 80GB the best choice?
Context Length and Throughput: For DeepSeek-R1:671b, what is the maximum context length and throughput that a single GPU can support?

3. Other Configurations:

GPU Memory and CPU Memory: After loading the model, what is the approximate GPU memory usage per card? Is there a need for additional CPU memory (e.g., 512GB DDR5) for large-scale batch processing?
Storage and Network: How demanding is the disk I/O when loading model weights? Would NVMe SSDs be recommended? If so, what are the suggested bandwidth specifications?
Multi-GPU Communication: Does multi-GPU communication require specific network configurations (e.g., InfiniBand or NVLink) to ensure optimal performance?

Thank you for providing the detailed technical insights and configuration advice. If there are any further documents or configuration guidelines, please let us know.

Originally created by @Zephyrsailor on GitHub (Feb 20, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/460 #### Description: We are planning to deploy the DeepSeek-R1 "full version" in a production environment and would like to confirm certain hardware and configuration requirements to ensure smooth deployment. Below are the key questions we have regarding the hardware configuration: ### 1. **Model Precision:** - **FP8 Precision:** Could you please confirm if the official "full version" uses FP8 precision? ### 2. **Hardware Configuration:** - **GPU Configuration:** For single-instance deployment, what are the recommended minimum and optimal GPU configurations? For example, is 8×A100 80GB the best choice? - **Context Length and Throughput:** For DeepSeek-R1:671b, what is the maximum context length and throughput that a single GPU can support? ### 3. **Other Configurations:** - **GPU Memory and CPU Memory:** After loading the model, what is the approximate GPU memory usage per card? Is there a need for additional CPU memory (e.g., 512GB DDR5) for large-scale batch processing? - **Storage and Network:** How demanding is the disk I/O when loading model weights? Would NVMe SSDs be recommended? If so, what are the suggested bandwidth specifications? - **Multi-GPU Communication:** Does multi-GPU communication require specific network configurations (e.g., InfiniBand or NVLink) to ensure optimal performance? Thank you for providing the detailed technical insights and configuration advice. If there are any further documents or configuration guidelines, please let us know.

bsakaguchi

2026-10-26 21:20:10 +00:00

closed this issue
added the
stale

closed-as-stale
labels

bsakaguchi commented

2026-10-26 21:20:11 +00:00

Author

Owner

@pp5ee commented on GitHub (Feb 27, 2025):

I am just curious about the speed of multi-GPU communication when it is doing the inference, we have an 18xGPU hardware solution (single rig, one motherboard) with PCIE4 or PCIE5, and we are about to test it, and once I have the result I can share it to you. And by the way, many people modify the vram of 4090 0r 3090 from 24g to 48G or even 90g(?) to fit the requirement of large param model inference, the key ,I think, is the communication speed.

@pp5ee commented on GitHub (Feb 27, 2025): I am just curious about the speed of multi-GPU communication when it is doing the inference, we have an 18xGPU hardware solution (single rig, one motherboard) with PCIE4 or PCIE5, and we are about to test it, and once I have the result I can share it to you. And by the way, many people modify the vram of 4090 0r 3090 from 24g to 48G or even 90g(?) to fit the requirement of large param model inference, the key ,I think, is the communication speed.

bsakaguchi commented

2026-10-26 21:20:11 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Apr 23, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

@github-actions[bot] commented on GitHub (Apr 23, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

bsakaguchi commented

2026-10-26 21:20:12 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (May 27, 2025):

false

@github-actions[bot] commented on GitHub (May 27, 2025): false