[PR #103] [MERGED] Grouped GEMM skip useless computation for unaligned Ms #249

New issue

Closed

opened 2026-10-26 21:15:19 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:15:19 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/103
Author: @yukuai26
Created: 5/20/2025
Status: ✅ Merged
Merged: 5/27/2025
Merged by: @LyricZhao

Base: main ← Head: remove-useless-computation

📝 Commits (10+)

ccca476 Grouped GEMM skip useless computation for unaligned Ms
db1f0b5 Update readme.md
dc89674 small typo
e7e38ed Rename variables
2c5ab83 Restore previous indent
c4e31d1 Format
1169f83 Refactor tests
a5373e4 Add SkipComputation types
81f906e Bug fixed
81de208 Format

📊 Changes

5 files changed (+106 additions, -93 deletions)

View changed files

📝 README.md (+1 -1)
📝 deep_gemm/include/deep_gemm/fp8_gemm.cuh (+44 -39)
📝 deep_gemm/include/deep_gemm/scheduler.cuh (+14 -3)
📝 deep_gemm/jit/compiler.py (+1 -1)
📝 tests/test_core.py (+46 -49)

📄 Description

Skip useless computation on M and reconstruct test_m_grouped_gemm_contiguous and test_m_grouped_gemm_masked'.

In test_m_grouped_gemm_contiguous, there is a speedup of 0-15 TFLOPS (Due to the larger expected_m_per_group, the speedup effect is not very significant).

In test_m_grouped_gemm_masked, there is a speedup of 10-20 TFLOPS.

For GEMM, incorporating this optimization may affect the efficiency of certain shapes. I tested with m = 4160 and observed some speedup, but when m = 4096, there is a 10-20 TFLOPS reduction in efficiency.

This scenario can be specifically optimized by adding a template parameter to check whether m is a multiple of block_m. However, since GEMM is not the current optimization focus, I have not implemented this enhancement yet.Instead, we have directly disabled this optimization and are currently not making changes to GEMM's efficiency.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/103 **Author:** [@yukuai26](https://github.com/yukuai26) **Created:** 5/20/2025 **Status:** ✅ Merged **Merged:** 5/27/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `remove-useless-computation` --- ### 📝 Commits (10+) - [`ccca476`](https://github.com/deepseek-ai/DeepGEMM/commit/ccca476ac471a30e3655af3a947c4302f0562133) Grouped GEMM skip useless computation for unaligned Ms - [`db1f0b5`](https://github.com/deepseek-ai/DeepGEMM/commit/db1f0b5a1cd32ca6d53f73f16e1a45a11c615d1e) Update readme.md - [`dc89674`](https://github.com/deepseek-ai/DeepGEMM/commit/dc89674f47f3991d54ac9ee88766994aa1aa4b0d) small typo - [`e7e38ed`](https://github.com/deepseek-ai/DeepGEMM/commit/e7e38ed2226005c1f10b870bb6754b68eb05a948) Rename variables - [`2c5ab83`](https://github.com/deepseek-ai/DeepGEMM/commit/2c5ab83c6ca5313e1f7f2b6168d8191b0e32f749) Restore previous indent - [`c4e31d1`](https://github.com/deepseek-ai/DeepGEMM/commit/c4e31d121b455a5bffbcd741943823b5a261217f) Format - [`1169f83`](https://github.com/deepseek-ai/DeepGEMM/commit/1169f83c3685613f5e156cd1984005dcedddc202) Refactor tests - [`a5373e4`](https://github.com/deepseek-ai/DeepGEMM/commit/a5373e4bbdbdb15a4de7b6a31f84489121fd1a98) Add `SkipComputation` types - [`81f906e`](https://github.com/deepseek-ai/DeepGEMM/commit/81f906ef76b6371eae497d09466e97d0f3b04a2f) Bug fixed - [`81de208`](https://github.com/deepseek-ai/DeepGEMM/commit/81de20843046b478640b5c156326361f7247872c) Format ### 📊 Changes **5 files changed** (+106 additions, -93 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+1 -1) 📝 `deep_gemm/include/deep_gemm/fp8_gemm.cuh` (+44 -39) 📝 `deep_gemm/include/deep_gemm/scheduler.cuh` (+14 -3) 📝 `deep_gemm/jit/compiler.py` (+1 -1) 📝 `tests/test_core.py` (+46 -49) </details> ### 📄 Description Skip useless computation on M and reconstruct test_m_grouped_gemm_contiguous and test_m_grouped_gemm_masked'. In test_m_grouped_gemm_contiguous, there is a speedup of 0-15 TFLOPS (Due to the larger expected_m_per_group, the speedup effect is not very significant). In test_m_grouped_gemm_masked, there is a speedup of 10-20 TFLOPS. For GEMM, incorporating this optimization may affect the efficiency of certain shapes. I tested with m = 4160 and observed some speedup, but when m = 4096, there is a 10-20 TFLOPS reduction in efficiency. This scenario can be specifically optimized by adding a template parameter to check whether m is a multiple of block_m. However, since GEMM is not the current optimization focus, I have not implemented this enhancement yet.Instead, we have directly disabled this optimization and are currently not making changes to GEMM's efficiency. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 21:15:19 +00:00