[PR #65] fix: bump flashinfer to 0.6.9 to match the V4-Flash MXFP4 minimum #65

New issue

Open

opened 2026-10-29 21:06:53 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:06:53 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/sglang/pull/65
Author: @gvr13n
Created: 7/12/2026
Status: 🔄 Open

Base: main ← Head: fix/flashinfer-min-version-v4

📝 Commits (1)

6485b99 fix: bump flashinfer to 0.6.9 to match the V4-Flash MXFP4 minimum

📊 Changes

2 files changed (+3 additions, -3 deletions)

View changed files

📝 docker/Dockerfile (+1 -1)
📝 python/pyproject.toml (+2 -2)

📄 Description

Fresh installs cannot serve DeepSeek-V4-Flash MXFP4: mxfp4_deepseek.py requires flashinfer >= 0.6.9 (_MIN_FI_VERSION — mxfp8_quantize, trtllm_fp4_block_scale_routed_moe), but pyproject.toml pins flashinfer_python/flashinfer_cubin ==0.6.3.

The failure mode is nasty to diagnose: the quant plugin fails its import-time version check with only a warning, the MXFP4 MoE method never registers, and the server later dies with an unrelated-looking AssertionError: Hidden size mismatch in the generic triton fused_experts during cuda-graph capture. (Hit this twice on a working V4-Flash + kt-hybrid box after a routine reinstall silently downgraded flashinfer per the pin.)

This bumps the pins and the Dockerfile FLASHINFER_VERSION arg together, preserving the documented jit-cache alignment.

Possibly worth a follow-up: promote the import-time version warning to a hard error when a V4 MXFP4 checkpoint is actually being served, so the failure surfaces at the true cause.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/sglang/pull/65 **Author:** [@gvr13n](https://github.com/gvr13n) **Created:** 7/12/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/flashinfer-min-version-v4` --- ### 📝 Commits (1) - [`6485b99`](https://github.com/kvcache-ai/sglang/commit/6485b99af467795deb22a97a0a7fe86d4ae8f9fe) fix: bump flashinfer to 0.6.9 to match the V4-Flash MXFP4 minimum ### 📊 Changes **2 files changed** (+3 additions, -3 deletions) <details> <summary>View changed files</summary> 📝 `docker/Dockerfile` (+1 -1) 📝 `python/pyproject.toml` (+2 -2) </details> ### 📄 Description Fresh installs cannot serve DeepSeek-V4-Flash MXFP4: `mxfp4_deepseek.py` requires flashinfer >= 0.6.9 (`_MIN_FI_VERSION` — `mxfp8_quantize`, `trtllm_fp4_block_scale_routed_moe`), but `pyproject.toml` pins `flashinfer_python`/`flashinfer_cubin` `==0.6.3`. The failure mode is nasty to diagnose: the quant plugin fails its import-time version check with only a warning, the MXFP4 MoE method never registers, and the server later dies with an unrelated-looking `AssertionError: Hidden size mismatch` in the generic triton `fused_experts` during cuda-graph capture. (Hit this twice on a working V4-Flash + kt-hybrid box after a routine reinstall silently downgraded flashinfer per the pin.) This bumps the pins and the Dockerfile `FLASHINFER_VERSION` arg together, preserving the documented jit-cache alignment. Possibly worth a follow-up: promote the import-time version warning to a hard error when a V4 MXFP4 checkpoint is actually being served, so the failure surfaces at the true cause. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-29 21:06:53 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/sglang#65

No description provided.

Rows
Columns