[PR #403] Fix SM120 scale-factor layout transformation #421

New issue

Open

opened 2026-10-26 21:17:35 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:17:35 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/403
Author: @bvolpato
Created: 8/8/2026
Status: 🔄 Open

Base: main ← Head: bvolpato/fix-sm120-sf-layout

📝 Commits (1)

9ac2b5b Fix SM120 scale-factor layout transformation

📊 Changes

2 files changed (+33 additions, -4 deletions)

View changed files

📝 csrc/apis/layout.hpp (+4 -4)
📝 tests/test_layout.py (+29 -0)

📄 Description

Summary

Addresses #372 by allowing explicit SM120 scale-factor recipes to use the existing SM100 UE8M0 packing and packed-scale validation paths. The same architecture support is already present in nv_dev.

Default recipes, K-grouped layouts, and broader SM120 inference support are outside this change. SM121 has not been tested.

Reproduction

On main at 057ca59, this fails on an RTX 5070 Ti with Unknown SF transformation:

import torch
import deep_gemm

sf = torch.ones((4, 128, 8), device="cuda", dtype=torch.float32)
deep_gemm.transform_sf_into_required_layout(
    sf, 128, 256, (1, 1, 32), num_groups=4, is_sfa=False,
)

With the patch, it returns packed torch.int32 scales with shape (4, 128, 2) and stride (256, 1, 128).

Testing

The build, reproduction, and layout checks passed on an RTX 5070 Ti with CUDA 13.2 and PyTorch 2.13.0+cu130:

./develop.sh
DG_USE_NVIDIA_TOOLS=1 PYTHONPATH=. python -m pytest -q tests/test_layout.py

DG_USE_NVIDIA_TOOLS=1 skips profiling while keeping correctness checks enabled; SM100-only cases did not run on this GPU. Builds used locally supplied elfutils development headers, and git diff --check passed.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/403 **Author:** [@bvolpato](https://github.com/bvolpato) **Created:** 8/8/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `bvolpato/fix-sm120-sf-layout` --- ### 📝 Commits (1) - [`9ac2b5b`](https://github.com/deepseek-ai/DeepGEMM/commit/9ac2b5b974ec5a95315fbff578d702cf2adc71c9) Fix SM120 scale-factor layout transformation ### 📊 Changes **2 files changed** (+33 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `csrc/apis/layout.hpp` (+4 -4) 📝 `tests/test_layout.py` (+29 -0) </details> ### 📄 Description ## Summary Addresses #372 by allowing explicit SM120 scale-factor recipes to use the existing SM100 UE8M0 packing and packed-scale validation paths. The same architecture support is already present in [`nv_dev`](https://github.com/deepseek-ai/DeepGEMM/commit/373f0738de7a320d71d1e7d1824f012a10416e8b). Default recipes, K-grouped layouts, and broader SM120 inference support are outside this change. SM121 has not been tested. ## Reproduction On `main` at `057ca59`, this fails on an RTX 5070 Ti with `Unknown SF transformation`: ```python import torch import deep_gemm sf = torch.ones((4, 128, 8), device="cuda", dtype=torch.float32) deep_gemm.transform_sf_into_required_layout( sf, 128, 256, (1, 1, 32), num_groups=4, is_sfa=False, ) ``` With the patch, it returns packed `torch.int32` scales with shape `(4, 128, 2)` and stride `(256, 1, 128)`. ## Testing The build, reproduction, and layout checks passed on an RTX 5070 Ti with CUDA 13.2 and PyTorch 2.13.0+cu130: ```bash ./develop.sh DG_USE_NVIDIA_TOOLS=1 PYTHONPATH=. python -m pytest -q tests/test_layout.py ``` `DG_USE_NVIDIA_TOOLS=1` skips profiling while keeping correctness checks enabled; SM100-only cases did not run on this GPU. Builds used locally supplied elfutils development headers, and `git diff --check` passed. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 21:17:35 +00:00

bsakaguchi referenced this issue

2026-10-26 21:17:44 +00:00

[PR #421] fix: make pack_ue8m0_to_int CUDA-graph capture safe (#414) #433