deepseek-ai/DeepGEMM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00
Code Issues 162 Projects Packages Wiki Activity

[GH-ISSUE #107] [Perf] Grouped W-Grad GEMM is 10–18 % slower than the released benchmark on H200 (#95) #61

New issue
Closed
opened 2026-10-26 21:11:34 +00:00 by bsakaguchi · 4 comments
bsakaguchi commented 2026-10-26 21:11:34 +00:00
Owner
Copy link

Originally created by @kaimo455 on GitHub (May 28, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/107

📌 Summary

grouped weight gradient gemm in PR #95 is markedly slower than the same workloads built from 78d8362e7a on an H200.
Depending on num_groups , M, N, K, I see –9 % → –18 % lower TFLOPS.

🖥️ Test environment

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:18:24_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0

torch.cuda.get_device_properties(device='cuda')
>>> _CudaDeviceProperties(name='NVIDIA H200', major=9, minor=0, total_memory=143071MB, multi_processor_count=132, uuid=xxx, L2_cache_size=60MB)

DeepGEMM commit ID: 78d8362e7a

Reproducer script
used the tests/test_core.py script.

📊 Results

Groups m n avg_k time (µs) H200 TFLOPS H200 GB/s PR #95 TFLOPS PR #95 GB/s Diff
4 7168 4096 4096 1183 813 354 939 409 -13 %
4 2048 7168 4096 591 814 454 900 502 -10 %
4 7168 4096 8192 2241 858 269 1044 328 -18 %
4 2048 7168 8192 1102 873 381 1033 450 -15 %
8 7168 4096 4096 2359 816 356 942 411 -13 %
8 2048 7168 4096 1167 825 460 902 503 -9 %

Thanks for any insight!

Originally created by @kaimo455 on GitHub (May 28, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/107 ### 📌 Summary `grouped weight gradient gemm` in **PR #95** is markedly slower than the same workloads built from 78d8362e7a7fded67e730a74c9423fd05db003dd on an H200. Depending on `num_groups , M, N, K`, I see –9 % → –18 % lower TFLOPS. --- ### 🖥️ Test environment `nvcc --version` > nvcc: NVIDIA (R) Cuda compiler driver > Copyright (c) 2005-2024 NVIDIA Corporation > Built on Thu_Mar_28_02:18:24_PDT_2024 > Cuda compilation tools, release 12.4, V12.4.131 > Build cuda_12.4.r12.4/compiler.34097967_0 ```python torch.cuda.get_device_properties(device='cuda') >>> _CudaDeviceProperties(name='NVIDIA H200', major=9, minor=0, total_memory=143071MB, multi_processor_count=132, uuid=xxx, L2_cache_size=60MB) ``` DeepGEMM commit ID: 78d8362e7a7fded67e730a74c9423fd05db003dd > **Reproducer script** > used the `tests/test_core.py` script. --- ### 📊 Results Groups | m | n | avg_k | time (µs) | H200 TFLOPS | H200 GB/s | PR #95 TFLOPS | PR #95 GB/s | Diff -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 4 | 7168 | 4096 | 4096 | 1183 | 813 | 354 | 939 | 409 | -13 % 4 | 2048 | 7168 | 4096 | 591 | 814 | 454 | 900 | 502 | -10 % 4 | 7168 | 4096 | 8192 | 2241 | 858 | 269 | 1044 | 328 | -18 % 4 | 2048 | 7168 | 8192 | 1102 | 873 | 381 | 1033 | 450 | -15 % 8 | 7168 | 4096 | 4096 | 2359 | 816 | 356 | 942 | 411 | -13 % 8 | 2048 | 7168 | 4096 | 1167 | 825 | 460 | 902 | 503 | -9 % Thanks for any insight!
bsakaguchi commented 2026-10-26 21:11:36 +00:00
Author
Owner
Copy link

@LyricZhao commented on GitHub (May 29, 2025):

"NVIDIA H200 (Hopper H100-SXMo-H200, 96 SM @ 1.94 GHz, 141 GB HBM3e)"

Why only 96 SMs?

<!-- gh-comment-id:2918391277 --> @LyricZhao commented on GitHub (May 29, 2025): "NVIDIA H200 (Hopper H100-SXMo-H200, 96 SM @ 1.94 GHz, 141 GB HBM3e)" Why only 96 SMs?
bsakaguchi commented 2026-10-26 21:11:37 +00:00
Author
Owner
Copy link

@kaimo455 commented on GitHub (May 30, 2025):

@LyricZhao There is a "copy&paste" mistake for that GPU information before, so I replace with nvcc informatin and other device information from torch.

<!-- gh-comment-id:2921523512 --> @kaimo455 commented on GitHub (May 30, 2025): @LyricZhao There is a "copy&paste" mistake for that GPU information before, so I replace with `nvcc` informatin and other device information from torch.
bsakaguchi commented 2026-10-26 21:11:37 +00:00
Author
Owner
Copy link

@LyricZhao commented on GitHub (May 30, 2025):

Can you please try CUDA 12.9? I guess the post-optimization is not working for wgrad before 12.9.

<!-- gh-comment-id:2921533072 --> @LyricZhao commented on GitHub (May 30, 2025): Can you please try CUDA 12.9? I guess the post-optimization is not working for wgrad before 12.9.
bsakaguchi commented 2026-10-26 21:11:38 +00:00
Author
Owner
Copy link

@kaimo455 commented on GitHub (Jun 3, 2025):

I upgraded to CUDA 12.9, and the performance now matches the PR’s numbers. 👍

<!-- gh-comment-id:2935278666 --> @kaimo455 commented on GitHub (Jun 3, 2025): I upgraded to CUDA 12.9, and the performance now matches the PR’s numbers. 👍
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
public-sync-20261014
0930-release
nv_dev
fix-mega-moe-fence
public-release-2609
public-release-2607
public-release-2606
mega-update
public-release-260416
ko3n1g/ci/build-wheels
nv_dev_f8e8fb5
nv_dev_950d31a
nv_dev_67fc648
nv_dev_c491439
nv_dev_4ff3f54
v2.1.1.post3
v2.1.1.post2
v2.1.1.post1
v2.1.1
v2.1.0
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepGEMM#61
No description provided.