[PR #158] [MERGED] Fix inappropriate configs for some small shapes #269

New issue

Closed

opened 2026-10-26 21:15:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:15:34 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/158
Author: @coconut902
Created: 8/12/2025
Status: ✅ Merged
Merged: 8/14/2025
Merged by: @LyricZhao

Base: main ← Head: jzh/polish_get_best_configs_modeling

📝 Commits (1)

48600f1 Polish get_best_configs modeling.

📊 Changes

1 file changed (+3 additions, -1 deletions)

View changed files

📝 csrc/jit_kernels/heuristics/common.hpp (+3 -1)

📄 Description

This PR polishes get_best_configs modeling, to avoid picking inappropriate (block_m, block_n) for some shapes.
Bad cases before this PR:

	(m, n, k)	picked (block_m, block_n, block_k)	expected best (block_m, block_n, block_k)	kernel duration (on H20)
sm90_fp8_gemm_1d2d	64, 16, 8192	256, 48, 128	64, 16, 128	58 us

After this PR, a larger block_n is picked only when block_m, block_n are smaller than shape m, n, respectively.
After this PR:

	(m, n, k)	picked (block_m, block_n, block_k)	expected best (block_m, block_n, block_k)	kernel duration (on H20)
sm90_fp8_gemm_1d2d	64, 16, 8192	64, 16, 128	64, 16, 128	12 us

Normal shapes (listed in tests/generators.py) are fully tested, and there is no performance degression.
It might also be a bug for some shapes on sm100. But I don't have sm100 gpus for testing this case.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/158 **Author:** [@coconut902](https://github.com/coconut902) **Created:** 8/12/2025 **Status:** ✅ Merged **Merged:** 8/14/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `jzh/polish_get_best_configs_modeling` --- ### 📝 Commits (1) - [`48600f1`](https://github.com/deepseek-ai/DeepGEMM/commit/48600f132c6507f8781ca287f257f598c7c7c350) Polish get_best_configs modeling. ### 📊 Changes **1 file changed** (+3 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `csrc/jit_kernels/heuristics/common.hpp` (+3 -1) </details> ### 📄 Description This PR polishes get_best_configs modeling, to avoid picking inappropriate `(block_m, block_n)` for some shapes. Bad cases before this PR: || (m, n, k) | picked (block_m, block_n, block_k) | expected best (block_m, block_n, block_k) | kernel duration (on H20) | |:---|:---|:---|:---|:---| | sm90_fp8_gemm_1d2d | 64, 16, 8192 | 256, 48, 128 | 64, 16, 128 | 58 us | After this PR, a larger `block_n` is picked only when `block_m`, `block_n` are smaller than shape `m`, `n`, respectively. After this PR: || (m, n, k) | picked (block_m, block_n, block_k) | expected best (block_m, block_n, block_k) | kernel duration (on H20) | |:---|:---|:---|:---|:---| | sm90_fp8_gemm_1d2d | 64, 16, 8192 | 64, 16, 128 | 64, 16, 128 | 12 us | Normal shapes (listed in tests/generators.py) are fully tested, and there is no performance degression. It might also be a bug for some shapes on sm100. But I don't have sm100 gpus for testing this case. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 21:15:34 +00:00