mirror of
https://github.com/deepseek-ai/DeepEP.git
synced 2026-11-05 00:47:32 +00:00
[GH-ISSUE #5] DeepEP 对使用 Co-Packaged Optics (CPO) 的系统有益，特别是那些需要高性能 AI 训练的场景 #6
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepEP#6
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Ted-MrYang on GitHub (Feb 25, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/5
Technical Overview of DeepEP
DeepEP provides high throughput and low latency all-to-all GPU kernels specifically for MoE dispatch and combination, supporting low-precision operations including FP8 (DeepEP GitHub page). It is associated with the training of large language models such as DeepSeek-V3, and provides kernels with asymmetric domain bandwidth forwarding (NVLink to RDMA) and SM count control, suitable for training and inference pre-population (DeepEP GitHub page). Its performance testing hardware includes H800 and CX7 InfiniBand 400 Gb/s RDMA, with common core bandwidth reaching 153-158 GB/s within the node and 43-47 GB/s between nodes (DeepEP GitHub page).
DeepEP’s requirements include Hopper GPUs, Python 3.8+, CUDA 12.3+, PyTorch 2.1+, NVLink, and RDMA (DeepEP GitHub page), which suggests that it relies on high-performance GPU cluster communications and is particularly suitable for distributed AI training.
Analysis of the relationship between DeepEP and CPO
Although DeepEP and CPO belong to different technology layers (software vs. hardware), they can work together in the same system. DeepEP focuses on optimizing communications in AI training, while CPO provides high-speed optical communication networks at the physical layer. In particular, CPO can serve as the network foundation for DeepEP to run, providing high bandwidth and low latency, which are very important for distributed computing of AI training.
For systems using CPO, DeepEP can leverage its high-speed network to optimize communications for AI training. For example, in a data center, CPO can support switch ASICs with over 51.2 TBps, which is very helpful for distributed training of DeepEP (Synopsys Blog). This means that DeepEP will be more efficient when running in such systems, especially in scenarios that require massively parallel computing.
Overall, DeepEP is beneficial to systems using Co-Packaged Optics (CPO), especially those scenarios that require high-performance AI training. It can optimize communications by utilizing the high-speed optical communication network provided by CPO. Therefore, the reference value of DeepEP's GitHub repository for CPO-related workers lies in understanding how the communication requirements of AI training affect network design.
`
DeepEP 的技术概述
DeepEP 提供高吞吐量和低延迟的全对全 GPU 内核，专门用于 MoE 分派和组合，支持包括 FP8 在内的低精度操作（DeepEP GitHub 页面）。它与 DeepSeek-V3 等大型语言模型的训练相关联，提供不对称域带宽转发（NVLink 到 RDMA）和 SM 数量控制的内核，适用于训练和推理预填充（DeepEP GitHub 页面）。其性能测试硬件包括 H800 和 CX7 InfiniBand 400 Gb/s RDMA，普通内核带宽节点内可达 153-158 GB/s，节点间 43-47 GB/s（DeepEP GitHub 页面）。
DeepEP 的要求包括 Hopper GPUs、Python 3.8+、CUDA 12.3+、PyTorch 2.1+、NVLink 和 RDMA（DeepEP GitHub 页面），这表明它依赖于高性能的 GPU 集群通信，特别适合分布式 AI 训练。
DeepEP 与 CPO 的关系分析
虽然 DeepEP 和 CPO 属于不同的技术层（软件 vs. 硬件），但它们可以在同一个系统中协同工作。DeepEP 专注于优化 AI 训练中的通信，而 CPO 提供物理层的高速光通信网络。特别是，CPO 可以作为 DeepEP 运行的网络基础，提供高带宽和低延迟，这对 AI 训练的分布式计算非常重要。
对于使用 CPO 的系统，DeepEP 可以利用其高速网络来优化 AI 训练的通信。例如，在数据中心中，CPO 可以支持超过 51.2 TBps 的交换机 ASIC，这对 DeepEP 的分布式训练非常有帮助（Synopsys Blog）。这意味着，DeepEP 在这样的系统中运行时会更高效，特别是在需要大规模并行计算的场景中。
综合来看，DeepEP 对使用 Co-Packaged Optics (CPO) 的系统有益，特别是那些需要高性能 AI 训练的场景。它可以利用 CPO 提供的高速光通信网络来优化通信。因此，DeepEP 的 GitHub 仓库对 CPO 相关工作者的参考价值在于了解 AI 训练的通信需求如何影响网络设计。
`
@cauherk commented on GitHub (Feb 25, 2025):
Cables are also a viable option with better cost-effectiveness.
@AIBionics commented on GitHub (Feb 25, 2025):
转人工
@Ted-MrYang commented on GitHub (Feb 25, 2025):
Copper cables perform outstandingly in terms of cost-effectiveness. In scenarios with limited budgets, they can be used to build a network architecture that meets basic data transmission requirements at a relatively low cost, thus having a high cost-performance ratio. However, they have relatively obvious heat generation issues during operation. Moreover, compared with other transmission media such as optical fibers or CPO (Co-packaged Optics), copper cables do not have an advantage in terms of the upper limits of transmission bandwidth and data transmission rate, and it is difficult for them to meet the application scenarios with stringent requirements for high-speed and large-capacity data transmission.