[PR #318] Support SM120 #362

New issue

Open

opened 2026-10-26 21:16:48 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:16:48 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/318
Author: @jasl
Created: 4/25/2026
Status: 🔄 Open

Base: main ← Head: sm120

📝 Commits (10+)

0711c29 Upgrade CUTLASS to 4.4.2
2206a1d Add SM120 reference fallbacks for V4 blockers
959f1df Add SM120 FP8 einsum reference fallback
23ec7b5 Exclude local Codex config
64978f1 Implement SM120 kernel for tf32_hc_prenorm_gemm
dd05d89 Update SM120 HC regression for kernel path
5355f51 Strengthen SM120 fallback regression coverage
6d624fb Vectorize SM120 FP8 paged MQA dot loop
47369c1 Vectorize SM120 FP8 einsum fallback
d863f7b Cover larger SM120 HC prenorm GEMM tiles

📊 Changes

17 files changed (+2035 additions, -8 deletions)

View changed files

📝 .gitignore (+4 -1)
📝 csrc/apis/attention.hpp (+8 -4)
📝 csrc/apis/einsum.hpp (+8 -0)
📝 csrc/apis/hyperconnection.hpp (+3 -0)
📝 csrc/indexing/main.cu (+1 -0)
📝 csrc/jit/compiler.hpp (+3 -2)
📝 csrc/jit/device_runtime.hpp (+5 -0)
➕ csrc/jit_kernels/heuristics/sm120.hpp (+243 -0)
➕ csrc/jit_kernels/impls/sm120_fp8_einsum.hpp (+149 -0)
➕ csrc/jit_kernels/impls/sm120_tf32_hc_prenorm_gemm.hpp (+154 -0)
📝 csrc/jit_kernels/impls/smxx_fp8_fp4_paged_mqa_logits.hpp (+205 -0)
➕ deep_gemm/include/deep_gemm/impls/sm120_fp8_einsum.cuh (+126 -0)
➕ deep_gemm/include/deep_gemm/impls/sm120_fp8_paged_mqa_logits.cuh (+395 -0)
➕ deep_gemm/include/deep_gemm/impls/sm120_tf32_hc_prenorm_gemm.cuh (+288 -0)
➕ deep_gemm/include/deep_gemm/mma/sm120.cuh (+62 -0)
➕ tests/test_sm120_kernels.py (+380 -0)
📝 third-party/cutlass (+1 -1)

📄 Description

Disclaimer: I'm not working in the AI domain, but I wish my RTX Pro 6000 could run DeepSeek v4, so I made this.
GPT is the co-worker.

With this PoC, with some other dirty hacks, I've made it run on my 2 x RTX Pro 6000 workstation.

See my vLLM PR https://github.com/vllm-project/vllm/pull/40991

If anyone can help, please take it.
I shall try my best to make it fully work, but it's a challenge to me.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/318 **Author:** [@jasl](https://github.com/jasl) **Created:** 4/25/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `sm120` --- ### 📝 Commits (10+) - [`0711c29`](https://github.com/deepseek-ai/DeepGEMM/commit/0711c291e43fd9042c8a1f743f95fef34876944c) Upgrade CUTLASS to 4.4.2 - [`2206a1d`](https://github.com/deepseek-ai/DeepGEMM/commit/2206a1d7bc6d4df81737d74efb6bacf5888f1fbb) Add SM120 reference fallbacks for V4 blockers - [`959f1df`](https://github.com/deepseek-ai/DeepGEMM/commit/959f1df759ed591cadb463ab9af68222428de5df) Add SM120 FP8 einsum reference fallback - [`23ec7b5`](https://github.com/deepseek-ai/DeepGEMM/commit/23ec7b5662101edbba60cf82a6661512a6759907) Exclude local Codex config - [`64978f1`](https://github.com/deepseek-ai/DeepGEMM/commit/64978f1ca9a51b3072eaf05a3b689813b657ab7f) Implement SM120 kernel for tf32_hc_prenorm_gemm - [`dd05d89`](https://github.com/deepseek-ai/DeepGEMM/commit/dd05d89b038165f1c85648d239c2b5ff14fd4ba7) Update SM120 HC regression for kernel path - [`5355f51`](https://github.com/deepseek-ai/DeepGEMM/commit/5355f5196429e009748f59d599fbf39c848ce263) Strengthen SM120 fallback regression coverage - [`6d624fb`](https://github.com/deepseek-ai/DeepGEMM/commit/6d624fb63346c2f4e83d075dcc75f112601a9cb0) Vectorize SM120 FP8 paged MQA dot loop - [`47369c1`](https://github.com/deepseek-ai/DeepGEMM/commit/47369c1d2df7ed36c685dec77cec217a90a94a18) Vectorize SM120 FP8 einsum fallback - [`d863f7b`](https://github.com/deepseek-ai/DeepGEMM/commit/d863f7bfd8c1e53a958a42dc5c62248d56062bff) Cover larger SM120 HC prenorm GEMM tiles ### 📊 Changes **17 files changed** (+2035 additions, -8 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+4 -1) 📝 `csrc/apis/attention.hpp` (+8 -4) 📝 `csrc/apis/einsum.hpp` (+8 -0) 📝 `csrc/apis/hyperconnection.hpp` (+3 -0) 📝 `csrc/indexing/main.cu` (+1 -0) 📝 `csrc/jit/compiler.hpp` (+3 -2) 📝 `csrc/jit/device_runtime.hpp` (+5 -0) ➕ `csrc/jit_kernels/heuristics/sm120.hpp` (+243 -0) ➕ `csrc/jit_kernels/impls/sm120_fp8_einsum.hpp` (+149 -0) ➕ `csrc/jit_kernels/impls/sm120_tf32_hc_prenorm_gemm.hpp` (+154 -0) 📝 `csrc/jit_kernels/impls/smxx_fp8_fp4_paged_mqa_logits.hpp` (+205 -0) ➕ `deep_gemm/include/deep_gemm/impls/sm120_fp8_einsum.cuh` (+126 -0) ➕ `deep_gemm/include/deep_gemm/impls/sm120_fp8_paged_mqa_logits.cuh` (+395 -0) ➕ `deep_gemm/include/deep_gemm/impls/sm120_tf32_hc_prenorm_gemm.cuh` (+288 -0) ➕ `deep_gemm/include/deep_gemm/mma/sm120.cuh` (+62 -0) ➕ `tests/test_sm120_kernels.py` (+380 -0) 📝 `third-party/cutlass` (+1 -1) </details> ### 📄 Description Disclaimer: I'm not working in the AI domain, but I wish my RTX Pro 6000 could run DeepSeek v4, so I made this. GPT is the co-worker. With this PoC, with some other dirty hacks, I've made it run on my 2 x RTX Pro 6000 workstation. See my vLLM PR https://github.com/vllm-project/vllm/pull/40991 If anyone can help, please take it. I shall try my best to make it fully work, but it's a challenge to me. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 21:16:48 +00:00

bsakaguchi referenced this issue

2026-10-26 21:17:11 +00:00

[PR #362] Feat: sm90 support mxfp8 fp8 groupgemm #392