[PR #206] [MERGED] Support UE8M0 data format. #465

New issue

Closed

opened 2026-10-26 10:01:05 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 10:01:05 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/206
Author: @shifangx
Created: 6/12/2025
Status: ✅ Merged
Merged: 6/12/2025
Merged by: @LyricZhao

Base: main ← Head: shifang/ue8m0

📝 Commits (1)

e767a3e support UE8M0 data format.

📊 Changes

14 files changed (+258 additions, -118 deletions)

View changed files

📝 csrc/deep_ep.cpp (+38 -19)
📝 csrc/deep_ep.hpp (+2 -1)
📝 csrc/kernels/CMakeLists.txt (+2 -2)
📝 csrc/kernels/api.cuh (+6 -3)
📝 csrc/kernels/internode.cu (+11 -4)
📝 csrc/kernels/internode_ll.cu (+48 -21)
📝 csrc/kernels/intranode.cu (+10 -2)
📝 csrc/kernels/utils.cuh (+37 -0)
📝 deep_ep/buffer.py (+16 -10)
➕ install.sh (+12 -0)
📝 tests/test_internode.py (+5 -0)
📝 tests/test_intranode.py (+1 -0)
📝 tests/test_low_latency.py (+67 -56)
📝 tests/utils.py (+3 -0)

📄 Description

Hi DeepEP Team,
This PR is submitted by NVIDIA and introduces support for UE8M0 (unsigned 8-bit exponent, 0-bit mantissa) data format.

Key Changes

Normal Mode:
- Add input arguments scale_token_stride and scale_hidden_stride, which are used to calculate the scale's offset in the buffer.
Low Latency Mode:
- Add input argument round_scale to indicate whether round the scaling factors into power of 2.
- Add input argument use_ue8m0 to indicate whether to use UE8M0 as scaling factor format (available only with round_scale=True).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/206 **Author:** [@shifangx](https://github.com/shifangx) **Created:** 6/12/2025 **Status:** ✅ Merged **Merged:** 6/12/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `shifang/ue8m0` --- ### 📝 Commits (1) - [`e767a3e`](https://github.com/deepseek-ai/DeepEP/commit/e767a3e190cc1167112e43eb5353db2fc35693c2) support UE8M0 data format. ### 📊 Changes **14 files changed** (+258 additions, -118 deletions) <details> <summary>View changed files</summary> 📝 `csrc/deep_ep.cpp` (+38 -19) 📝 `csrc/deep_ep.hpp` (+2 -1) 📝 `csrc/kernels/CMakeLists.txt` (+2 -2) 📝 `csrc/kernels/api.cuh` (+6 -3) 📝 `csrc/kernels/internode.cu` (+11 -4) 📝 `csrc/kernels/internode_ll.cu` (+48 -21) 📝 `csrc/kernels/intranode.cu` (+10 -2) 📝 `csrc/kernels/utils.cuh` (+37 -0) 📝 `deep_ep/buffer.py` (+16 -10) ➕ `install.sh` (+12 -0) 📝 `tests/test_internode.py` (+5 -0) 📝 `tests/test_intranode.py` (+1 -0) 📝 `tests/test_low_latency.py` (+67 -56) 📝 `tests/utils.py` (+3 -0) </details> ### 📄 Description Hi DeepEP Team, This PR is submitted by NVIDIA and introduces support for UE8M0 (unsigned 8-bit exponent, 0-bit mantissa) data format. # Key Changes 1. **Normal Mode**: - Add input arguments `scale_token_stride` and `scale_hidden_stride`, which are used to calculate the scale's offset in the buffer. 2. **Low Latency Mode**: - Add input argument `round_scale` to indicate whether round the scaling factors into power of 2. - Add input argument `use_ue8m0` to indicate whether to use UE8M0 as scaling factor format (available only with `round_scale=True`). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 10:01:05 +00:00