[PR #450] feat(gemm): opt-in labels-contract checker for m-grouped contiguous GEMM #454

New issue

Open

opened 2026-10-26 21:18:02 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:18:02 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/450
Author: @lucifer1004
Created: 9/16/2026
Status: 🔄 Open

Base: main ← Head: fix/m-grouped-contiguous-labels-contract

📝 Commits (1)

ba1bcf5 feat(gemm): opt-in labels-contract checker for m-grouped contiguous GEMM

📊 Changes

5 files changed (+223 additions, -3 deletions)

View changed files

📝 README.md (+1 -0)
📝 csrc/apis/gemm.hpp (+37 -0)
📝 tests/generators.py (+7 -2)
📝 tests/test_bf16.py (+57 -0)
📝 tests/test_fp8_fp4.py (+121 -1)

📄 Description

Summary

Add an opt-in host-side contract checker for labels-mode m-grouped contiguous GEMM (m_grouped_fp8_fp4_gemm_nt_contiguous, m_grouped_bf16_gemm_nt_contiguous, and the nn forwarders), turning a silent wrong-results contract violation into a loud, actionable error.

The contract and the silent failure

The contiguous kernels select B (and SFB) per BLOCK_M tile from the label of the tile's first row (grouped_layout[m_block_idx * BLOCK_M]). The tile heuristics guarantee BLOCK_M divides the runtime mk_alignment_for_contiguous_layout, so labels whose group boundaries are aligned to the runtime alignment can never put two groups inside one tile.

A caller that builds labels at a finer granularity than the current runtime alignment (e.g. labels packed at 64 while the runtime default is 128) puts group boundaries inside a tile. The tile then computes all its rows with the first row's group B — silently corrupting every group after the first misaligned boundary. This was reported downstream as "an empty middle group corrupts later groups" (an empty group merely makes a misaligned boundary much more likely; no empty group is required — [128, 130, 65] built at 64 with runtime 128 corrupts the same way).

This is a caller-contract violation, and today it fails silently on every arch (sm90/sm100/sm12x all select B per tile from the first row's label).

What this PR does

csrc/apis/gemm.hpp: check_contiguous_labels_contract() — when DG_CHECK_CONTIGUOUS_LABELS is set, validates on the host (one GPU→CPU copy of the labels) that every group's first row starts at a multiple of the runtime mk alignment, and that labels are in range; violations raise a DGException naming the group, row, and alignment, with the remedy. Opt-in: zero cost in production.
tests/generators.py: generate_m_grouped_contiguous gains an optional actual_ms_override (default path byte-identical).
tests/test_fp8_fp4.py, tests/test_bf16.py:
- test_m_grouped_gemm_contiguous_middle_empty_groups — empty groups in the middle followed by non-empty groups (e.g. [100, 0, 130, 65]), the previously uncovered pattern class, verified contract-respecting;
- test_m_grouped_gemm_contiguous_labels_contract_rejection — misaligned labels raise loudly with the checker on, and pass with the checker off plus matching alignment.

Validation

sm_103 (B300): new tests pass; pre-existing test_m_grouped_gemm_contiguous (fp8_fp4 + bf16) and test_m_grouped_gemm_masked suites pass unchanged (the checker is inert unless enabled).
sm_120a: rejection path exercised (fires before arch dispatch; main has no sm12x dispatch by design).
No behavioral change with the env unset; no device-code change.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/450 **Author:** [@lucifer1004](https://github.com/lucifer1004) **Created:** 9/16/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/m-grouped-contiguous-labels-contract` --- ### 📝 Commits (1) - [`ba1bcf5`](https://github.com/deepseek-ai/DeepGEMM/commit/ba1bcf5a493076446d30d78d9596f503d4c739ba) feat(gemm): opt-in labels-contract checker for m-grouped contiguous GEMM ### 📊 Changes **5 files changed** (+223 additions, -3 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+1 -0) 📝 `csrc/apis/gemm.hpp` (+37 -0) 📝 `tests/generators.py` (+7 -2) 📝 `tests/test_bf16.py` (+57 -0) 📝 `tests/test_fp8_fp4.py` (+121 -1) </details> ### 📄 Description ## Summary Add an opt-in host-side contract checker for labels-mode m-grouped contiguous GEMM (`m_grouped_fp8_fp4_gemm_nt_contiguous`, `m_grouped_bf16_gemm_nt_contiguous`, and the `nn` forwarders), turning a silent wrong-results contract violation into a loud, actionable error. ## The contract and the silent failure The contiguous kernels select B (and SFB) per `BLOCK_M` tile from the label of the tile's **first row** (`grouped_layout[m_block_idx * BLOCK_M]`). The tile heuristics guarantee `BLOCK_M` divides the runtime `mk_alignment_for_contiguous_layout`, so labels whose group boundaries are aligned to the runtime alignment can never put two groups inside one tile. A caller that builds labels at a **finer** granularity than the current runtime alignment (e.g. labels packed at 64 while the runtime default is 128) puts group boundaries inside a tile. The tile then computes all its rows with the first row's group B — silently corrupting every group after the first misaligned boundary. This was reported downstream as "an empty middle group corrupts later groups" (an empty group merely makes a misaligned boundary much more likely; no empty group is required — `[128, 130, 65]` built at 64 with runtime 128 corrupts the same way). This is a caller-contract violation, and today it fails silently on every arch (sm90/sm100/sm12x all select B per tile from the first row's label). ## What this PR does - `csrc/apis/gemm.hpp`: `check_contiguous_labels_contract()` — when `DG_CHECK_CONTIGUOUS_LABELS` is set, validates on the host (one GPU→CPU copy of the labels) that every group's first row starts at a multiple of the runtime mk alignment, and that labels are in range; violations raise a `DGException` naming the group, row, and alignment, with the remedy. Opt-in: zero cost in production. - `tests/generators.py`: `generate_m_grouped_contiguous` gains an optional `actual_ms_override` (default path byte-identical). - `tests/test_fp8_fp4.py`, `tests/test_bf16.py`: - `test_m_grouped_gemm_contiguous_middle_empty_groups` — empty groups in the middle followed by non-empty groups (e.g. `[100, 0, 130, 65]`), the previously uncovered pattern class, verified contract-respecting; - `test_m_grouped_gemm_contiguous_labels_contract_rejection` — misaligned labels raise loudly with the checker on, and pass with the checker off plus matching alignment. ## Validation - sm_103 (B300): new tests pass; pre-existing `test_m_grouped_gemm_contiguous` (fp8_fp4 + bf16) and `test_m_grouped_gemm_masked` suites pass unchanged (the checker is inert unless enabled). - sm_120a: rejection path exercised (fires before arch dispatch; main has no sm12x dispatch by design). - No behavioral change with the env unset; no device-code change. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 21:18:02 +00:00

bsakaguchi referenced this issue

2026-10-26 21:18:06 +00:00

[PR #454] Add SM90 NVFP4+FP8 MegaMoE support #460