deepseek-ai/DeepGEMM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00
Code Issues 162 Projects Packages Wiki Activity

[GH-ISSUE #40] support for e5m2 FP8 #26

New issue
Closed
opened 2026-10-26 21:10:50 +00:00 by bsakaguchi · 1 comment
bsakaguchi commented 2026-10-26 21:10:50 +00:00
Owner
Copy link

Originally created by @TheRainstorm on GitHub (Mar 5, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/40

Hello, respected developer.

In the current code (such as fp8_gemm.cuh:84), the e4m3 format is directly used. Is the e5m2 FP8 format supported? Additionally, is it supported to multiply the left-hand side (lhs) in the e5m2 format by the right-hand side (rhs) in the e4m3 format? cuBLAS supports this operation.

Originally created by @TheRainstorm on GitHub (Mar 5, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/40 Hello, respected developer. In the current code (such as [fp8_gemm.cuh:84](https://github.com/deepseek-ai/DeepGEMM/blob/68fc742572ff50a3350eaeb0dbce58398a32cf49/deep_gemm/include/deep_gemm/fp8_gemm.cuh#L84C4-L84C39)), the e4m3 format is directly used. Is the e5m2 FP8 format supported? Additionally, is it supported to multiply the left-hand side (lhs) in the e5m2 format by the right-hand side (rhs) in the e4m3 format? cuBLAS supports this operation.
bsakaguchi commented 2026-10-26 21:10:51 +00:00
Author
Owner
Copy link

@soundOfDestiny commented on GitHub (Mar 5, 2025):

Hello, respected developer.

In the current code (such as fp8_gemm.cuh:84), the e4m3 format is directly used. Is the e5m2 FP8 format supported? Additionally, is it supported to multiply the left-hand side (lhs) in the e5m2 format by the right-hand side (rhs) in the e4m3 format? cuBLAS supports this operation.

We don't support e5m2 because it isn't used in DeepSeek V3/R1.
From another perspective, since e5m2 FP8 format has a much larger range than e4m3, perhaps it's unnecessary to use a very fine-grained scaling for common use.
Anyway, e5m2 GEMM can be implemented similarly. Besides replacing e4m3 with e5m2 and changing cast ops, it's important to use a WGMMA op of e5m2.

<!-- gh-comment-id:2700267099 --> @soundOfDestiny commented on GitHub (Mar 5, 2025): > Hello, respected developer. > > In the current code (such as [fp8_gemm.cuh:84](https://github.com/deepseek-ai/DeepGEMM/blob/68fc742572ff50a3350eaeb0dbce58398a32cf49/deep_gemm/include/deep_gemm/fp8_gemm.cuh#L84C4-L84C39)), the e4m3 format is directly used. Is the e5m2 FP8 format supported? Additionally, is it supported to multiply the left-hand side (lhs) in the e5m2 format by the right-hand side (rhs) in the e4m3 format? cuBLAS supports this operation. We don't support e5m2 because it isn't used in DeepSeek V3/R1. From another perspective, since e5m2 FP8 format has a much larger range than e4m3, perhaps it's unnecessary to use a very fine-grained scaling for common use. Anyway, e5m2 GEMM can be implemented similarly. Besides replacing e4m3 with e5m2 and changing cast ops, it's important to use a WGMMA op of e5m2.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
public-sync-20261014
0930-release
nv_dev
fix-mega-moe-fence
public-release-2609
public-release-2607
public-release-2606
mega-update
public-release-260416
ko3n1g/ci/build-wheels
nv_dev_f8e8fb5
nv_dev_950d31a
nv_dev_67fc648
nv_dev_c491439
nv_dev_4ff3f54
v2.1.1.post3
v2.1.1.post2
v2.1.1.post1
v2.1.1
v2.1.0
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepGEMM#26
No description provided.