[PR #517] Add internode Support for the SM80 architecture #606

New issue

Open

opened 2026-10-27 21:04:29 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:04:29 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/517
Author: @kzlxd
Created: 12/10/2025
Status: 🔄 Open

Base: main ← Head: SM80-Adaptation

📝 Commits (1)

e0bc51e Support internode for the SM80 architecture

📊 Changes

8 files changed (+518 additions, -79 deletions)

View changed files

📝 csrc/kernels/configs.cuh (+27 -4)
📝 csrc/kernels/internode.cu (+261 -21)
📝 csrc/kernels/internode_ll.cu (+118 -3)
📝 csrc/kernels/launch.cuh (+9 -17)
📝 csrc/kernels/utils.cuh (+18 -0)
📝 setup.py (+2 -2)
📝 tests/test_internode.py (+67 -27)
📝 tests/test_low_latency.py (+16 -5)

📄 Description

Background:

1). The A800 DeepEP only supports intra-machine communication; inter-machine communication is not supported;
2). We require multi-machine communication via DeepEP on our A800 systems.

Modifications

1). Supports compilation in internode mode under the SM80 architecture using:

export TORCH_CUDA_ARCH_LIST="8.0"
export DISABLE_SM90_FEATURES=1

2). Modified PTX instructions unsupported on the SM80 architecture,
e.g. atom.acquire.cta.shared::cta.cas.b32
3). Modified launch method to support cg::this_grid().sync() under SM80.
4). Employed the macro DISABLE_SM90_FEATURES to differentiate between SM90 and SM80 data copying approaches.

Performance testing

NVLINK bandwidth: 25GB/s * 8
NET: 100Gb/s * 8
Use the test script's default parameters directly, noting that the --disable-sm90-features parameter must be added. For normal mode, additionally include --test-ll-compatibility (omitting this on A800 will limit usage to only 4 network cards).

Normal

Type | Dispatch #EP | Bottleneck bandwidth | Combine #EP | Bottleneck bandwidth -- | -- | -- | -- | -- Intranode | 8 | 170.96 GB/s (NVLink) | 8 | 166.52 GB/s (NVLink) Internode | 16 | 21.22 GB/s (RDMA) | 16 | 20.59 GB/s (RDMA) Internode | 32 | 14.59 GB/s (RDMA) | 32 | 14.34 GB/s (RDMA)

Low-latency

Dispatch #EP | Latency | RDMA bandwidth | Combine #EP | Latency | RDMA bandwidth -- | -- | -- | -- | -- | -- 16 | 410.02 us | 18.32 GB/s | 16 | 728.45 us | 19.96 GB/s 32 | 672.17 us | 12.55 GB/s | 32 | 1174.00 us | 12.67 GB/s

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/517 **Author:** [@kzlxd](https://github.com/kzlxd) **Created:** 12/10/2025 **Status:** 🔄 Open **Base:** `main` ← **Head:** `SM80-Adaptation` --- ### 📝 Commits (1) - [`e0bc51e`](https://github.com/deepseek-ai/DeepEP/commit/e0bc51eb275b828d9aed00be686e7d436a8a9b90) Support internode for the SM80 architecture ### 📊 Changes **8 files changed** (+518 additions, -79 deletions) <details> <summary>View changed files</summary> 📝 `csrc/kernels/configs.cuh` (+27 -4) 📝 `csrc/kernels/internode.cu` (+261 -21) 📝 `csrc/kernels/internode_ll.cu` (+118 -3) 📝 `csrc/kernels/launch.cuh` (+9 -17) 📝 `csrc/kernels/utils.cuh` (+18 -0) 📝 `setup.py` (+2 -2) 📝 `tests/test_internode.py` (+67 -27) 📝 `tests/test_low_latency.py` (+16 -5) </details> ### 📄 Description ### Background: 1). The A800 DeepEP only supports intra-machine communication; inter-machine communication is not supported; 2). We require multi-machine communication via DeepEP on our A800 systems. ### Modifications 1). Supports compilation in internode mode under the SM80 architecture using: ``` export TORCH_CUDA_ARCH_LIST="8.0" export DISABLE_SM90_FEATURES=1 ``` 2). Modified PTX instructions unsupported on the SM80 architecture, e.g. `atom.acquire.cta.shared::cta.cas.b32` 3). Modified launch method to support cg::this_grid().sync() under SM80. 4). Employed the macro DISABLE_SM90_FEATURES to differentiate between SM90 and SM80 data copying approaches. ### Performance testing NVLINK bandwidth: 25GB/s * 8 NET: 100Gb/s * 8 Use the test script's default parameters directly, noting that the --disable-sm90-features parameter must be added. For normal mode, additionally include --test-ll-compatibility (omitting this on A800 will limit usage to only 4 network cards). <meta charset="UTF-8"><div class="mp-paragraph-wrapper hxj" data-morpho-type="paragraph" style="padding-left:0px;--indent-pixels:0px" data-slate-node="element" data-morpho-padding="0"><span data-morpho-text="Normal">Normal</span><div class="mp-paragraph-block-selection"></div></div><div class="mp-table-align"><div class="mp-table-wrapper"><div data-slate-node="element" style="padding-left:0px" class="mp-table-container"> Type | Dispatch #EP | Bottleneck bandwidth | Combine #EP | Bottleneck bandwidth -- | -- | -- | -- | -- Intranode | 8 | 170.96 GB/s (NVLink) | 8 | 166.52 GB/s (NVLink) Internode | 16 | 21.22 GB/s (RDMA) | 16 | 20.59 GB/s (RDMA) Internode | 32 | 14.59 GB/s (RDMA) | 32 | 14.34 GB/s (RDMA) </div><div class="mp-table-serialize-flag"><br/></div></div></div><span data-morpho-doc-data='{"token":"eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIiwiYXBwSWQiOjEsInVpZCI6IkxfcWM3dVFiT3oiLCJkb2NJZCI6IktOcWI4OEwyMV9fQS03In0..5dw15Di2MHYAAPJu.0zKEpS2fnNxhEACvv4p-jfLjv5VobT-hRTUTkxBTgsry7WdWIV9jVqOuFneC8E1QjqgX04RTLMIjc3kM3hRjSwvFkmYHMEQr2nibqIQNFAwN6GceV8vPJ6_mpTQxBByApOg1t5YquR8saACWYKfJXDddIA69Lcp9qIUPB8ekJKNfZ53bFQRXdSLT4VHkjjsy0eEzpf7FbK1dOATQiY1QUww6iw.LgrwKrEQszimFypWOtiAGA","appId":"1"}' class='mp-morpho-clipboard-doc-data' ></span> <meta charset="UTF-8"><div class="mp-paragraph-wrapper hxj" data-morpho-type="paragraph" style="padding-left:0px;--indent-pixels:0px" data-slate-node="element" data-morpho-padding="0"><span data-morpho-text="Low-latency">Low-latency</span><div class="mp-paragraph-block-selection"></div></div><div class="mp-table-align"><div class="mp-table-wrapper"><div data-slate-node="element" style="padding-left:0px" class="mp-table-container"> Dispatch #EP | Latency | RDMA bandwidth | Combine #EP | Latency | RDMA bandwidth -- | -- | -- | -- | -- | -- 16 | 410.02 us | 18.32 GB/s | 16 | 728.45 us | 19.96 GB/s 32 | 672.17 us | 12.55 GB/s | 32 | 1174.00 us | 12.67 GB/s </div><div class="mp-table-serialize-flag"><br/></div></div></div><span data-morpho-doc-data='{"token":"eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIiwiYXBwSWQiOjEsInVpZCI6IkxfcWM3dVFiT3oiLCJkb2NJZCI6IktOcWI4OEwyMV9fQS03In0..5dw15Di2MHYAAPJu.0zKEpS2fnNxhEACvv4p-jfLjv5VobT-hRTUTkxBTgsry7WdWIV9jVqOuFneC8E1QjqgX04RTLMIjc3kM3hRjSwvFkmYHMEQr2nibqIQNFAwN6GceV8vPJ6_mpTQxBByApOg1t5YquR8saACWYKfJXDddIA69Lcp9qIUPB8ekJKNfZ53bFQRXdSLT4VHkjjsy0eEzpf7FbK1dOATQiY1QUww6iw.LgrwKrEQszimFypWOtiAGA","appId":"1"}' class='mp-morpho-clipboard-doc-data' ></span> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-27 21:04:29 +00:00

bsakaguchi referenced this issue

2026-10-27 21:04:57 +00:00

[PR #606] fix build error without GPU for doca/NCCL #657

bsakaguchi referenced this issue

2026-10-27 21:05:01 +00:00

[PR #616] [MERGED] Fix DOCA build/path regressions from NIXL integration (#606) #665

bsakaguchi referenced this issue

2026-10-27 21:05:08 +00:00