[PR #458] [MERGED] Support hidden-dim 3072 for gpt-oss + low_latency kernels #581

New issue

Closed

opened 2026-10-27 21:04:16 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:04:16 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/458
Author: @varun-sundar-rabindranath
Created: 10/16/2025
Status: ✅ Merged
Merged: 10/17/2025
Merged by: @sphish

Base: main ← Head: varun/gptoss-padding

📝 Commits (1)

831f3fa support hidden-dim 3072

📊 Changes

1 file changed (+2 additions, -0 deletions)

View changed files

📝 csrc/kernels/launch.cuh (+2 -0)

📄 Description

gpt-oss model has a hidden-dim of 2880. At the moment, to transmit tensors of [M, 2880] with DeepEP low_latency kernels , we pad it up to 4096 which is too big of a padding.

This PR adds 3072 to the list of low_latency kernels so we may pad up to only 3072 during model execution.

Test

python3 test_low_latency.py --num-processes=2 --hidden 3072
Reserved 2 GPU(s): [7 3] for command execution
Allocating buffer size: 920.127872 MB ...
/dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/modules/transport/ibrc/ibrc.cpp:nvshmemt_init:1935: neither nv_peer_mem, or nvidia_peermem detected. Skipping transport.

/dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/modules/transport/ibgda/ibgda.cpp:nvshmemt_init:3630: neither nv_peer_mem, or nvidia_peermem detected. Skipping transport.

/dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/host/transport/transport.cpp:nvshmemi_transport_init:282: init failed for transport: IBGDA
/dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/modules/transport/ibrc/ibrc.cpp:nvshmemt_init:1935: neither nv_peer_mem, or nvidia_peermem detected. Skipping transport.

/dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/modules/transport/ibgda/ibgda.cpp:nvshmemt_init:3630: neither nv_peer_mem, or nvidia_peermem detected. Skipping transport.

/dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/host/transport/transport.cpp:nvshmemi_transport_init:282: init failed for transport: IBGDA
[rank 0] Dispatch + combine bandwidth: 102.71 GB/s, avg_t=92.09 us, min_t=89.79 us, max_t=94.69 us
[rank 1] Dispatch + combine bandwidth: 102.51 GB/s, avg_t=92.27 us, min_t=89.95 us, max_t=94.46 us
[rank 1] Dispatch bandwidth: 71.05 GB/s, avg_t=45.44 us | Combine bandwidth: 142.25 GB/s, avg_t=43.80 us
[rank 0] Dispatch bandwidth: 68.22 GB/s, avg_t=47.33 us | Combine bandwidth: 148.64 GB/s, avg_t=41.91 us
[rank 1] Dispatch send/recv time: 26.28 + 6.98 us | Combine send/recv time: 21.07 + 9.52 us
[rank 0] Dispatch send/recv time: 43.67 + 7.09 us | Combine send/recv time: 36.00 + 9.58 us

Ref : vLLM PR using gpt-oss with DeepEP Low Latency kernels https://github.com/vllm-project/vllm/pull/25997

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/458 **Author:** [@varun-sundar-rabindranath](https://github.com/varun-sundar-rabindranath) **Created:** 10/16/2025 **Status:** ✅ Merged **Merged:** 10/17/2025 **Merged by:** [@sphish](https://github.com/sphish) **Base:** `main` ← **Head:** `varun/gptoss-padding` --- ### 📝 Commits (1) - [`831f3fa`](https://github.com/deepseek-ai/DeepEP/commit/831f3fab7ba4b0b8a3f220b416988ea9763d133a) support hidden-dim 3072 ### 📊 Changes **1 file changed** (+2 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `csrc/kernels/launch.cuh` (+2 -0) </details> ### 📄 Description gpt-oss model has a hidden-dim of 2880. At the moment, to transmit tensors of [M, 2880] with DeepEP low_latency kernels , we pad it up to 4096 which is too big of a padding. This PR adds `3072` to the list of low_latency kernels so we may pad up to only `3072` during model execution. ## Test ``` python3 test_low_latency.py --num-processes=2 --hidden 3072 Reserved 2 GPU(s): [7 3] for command execution Allocating buffer size: 920.127872 MB ... /dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/modules/transport/ibrc/ibrc.cpp:nvshmemt_init:1935: neither nv_peer_mem, or nvidia_peermem detected. Skipping transport. /dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/modules/transport/ibgda/ibgda.cpp:nvshmemt_init:3630: neither nv_peer_mem, or nvidia_peermem detected. Skipping transport. /dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/host/transport/transport.cpp:nvshmemi_transport_init:282: init failed for transport: IBGDA /dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/modules/transport/ibrc/ibrc.cpp:nvshmemt_init:1935: neither nv_peer_mem, or nvidia_peermem detected. Skipping transport. /dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/modules/transport/ibgda/ibgda.cpp:nvshmemt_init:3630: neither nv_peer_mem, or nvidia_peermem detected. Skipping transport. /dvs/p4/build/sw/rel/gpgpu/toolkit/r12.8/main_nvshmem/src/host/transport/transport.cpp:nvshmemi_transport_init:282: init failed for transport: IBGDA [rank 0] Dispatch + combine bandwidth: 102.71 GB/s, avg_t=92.09 us, min_t=89.79 us, max_t=94.69 us [rank 1] Dispatch + combine bandwidth: 102.51 GB/s, avg_t=92.27 us, min_t=89.95 us, max_t=94.46 us [rank 1] Dispatch bandwidth: 71.05 GB/s, avg_t=45.44 us | Combine bandwidth: 142.25 GB/s, avg_t=43.80 us [rank 0] Dispatch bandwidth: 68.22 GB/s, avg_t=47.33 us | Combine bandwidth: 148.64 GB/s, avg_t=41.91 us [rank 1] Dispatch send/recv time: 26.28 + 6.98 us | Combine send/recv time: 21.07 + 9.52 us [rank 0] Dispatch send/recv time: 43.67 + 7.09 us | Combine send/recv time: 36.00 + 9.58 us ``` Ref : vLLM PR using gpt-oss with DeepEP Low Latency kernels https://github.com/vllm-project/vllm/pull/25997 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-27 21:04:16 +00:00