[PR #71] fix: five small guards (NextN post-load, MLA draft warmup hook, zero-expert Marlin, deep_gemm) #73

New issue

Open

opened 2026-10-29 21:06:57 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:06:57 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/sglang/pull/71
Author: @architehc
Created: 7/23/2026
Status: 🔄 Open

Base: main ← Head: sm120-misc-guards

📝 Commits (1)

6d9f7c5 fix: five small guards for NextN/MLA-draft/Marlin/deep_gemm edge paths

📊 Changes

5 files changed (+43 additions, -4 deletions)

View changed files

📝 python/sglang/srt/layers/attention/flashinfer_mla_backend.py (+4 -0)
📝 python/sglang/srt/layers/deep_gemm_wrapper/entrypoint.py (+3 -1)
📝 python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16_moe.py (+8 -0)
📝 python/sglang/srt/models/deepseek_common/deepseek_weight_loader.py (+19 -1)
📝 python/sglang/srt/models/deepseek_nextn.py (+9 -2)

📄 Description

Five independent small fixes found while serving GLM-5.2 with kt-kernel CPU experts:

NextN draft model: `run_post_load` passthrough (post-load hooks never ran for the draft weights).
Deepseek weight loader: fused `weight_shape` metadata merge (loading MTP checkpoints with fused shards).
`FlashInferMLAMultiStepDraftBackend`: `on_after_cuda_graph_warmup_pass` hook so draft backends can finalize after warmup.
Marlin MoE repack: skip when a layer has zero GPU experts (crash when all experts are CPU-offloaded).
`configure_deep_gemm_num_sms`: no-op when deep_gemm was never imported (NameError on the dual-stream decode path with JIT deepgemm disabled).

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/sglang/pull/71 **Author:** [@architehc](https://github.com/architehc) **Created:** 7/23/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `sm120-misc-guards` --- ### 📝 Commits (1) - [`6d9f7c5`](https://github.com/kvcache-ai/sglang/commit/6d9f7c5ad33d43f44d5c28880d398279d24ef4c7) fix: five small guards for NextN/MLA-draft/Marlin/deep_gemm edge paths ### 📊 Changes **5 files changed** (+43 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `python/sglang/srt/layers/attention/flashinfer_mla_backend.py` (+4 -0) 📝 `python/sglang/srt/layers/deep_gemm_wrapper/entrypoint.py` (+3 -1) 📝 `python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16_moe.py` (+8 -0) 📝 `python/sglang/srt/models/deepseek_common/deepseek_weight_loader.py` (+19 -1) 📝 `python/sglang/srt/models/deepseek_nextn.py` (+9 -2) </details> ### 📄 Description Five independent small fixes found while serving GLM-5.2 with kt-kernel CPU experts: 1. NextN draft model: \`run_post_load\` passthrough (post-load hooks never ran for the draft weights). 2. Deepseek weight loader: fused \`weight_shape\` metadata merge (loading MTP checkpoints with fused shards). 3. \`FlashInferMLAMultiStepDraftBackend\`: \`on_after_cuda_graph_warmup_pass\` hook so draft backends can finalize after warmup. 4. Marlin MoE repack: skip when a layer has zero GPU experts (crash when all experts are CPU-offloaded). 5. \`configure_deep_gemm_num_sms\`: no-op when deep_gemm was never imported (NameError on the dual-stream decode path with JIT deepgemm disabled). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-29 21:06:57 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/sglang#73

No description provided.

Rows
Columns