[GH-ISSUE #1859] SGLang main passes `num_gpu_experts` but kt-kernel expects `gpu_experts_mask` #1157

New issue

Closed

opened 2026-10-29 14:49:22 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:49:22 +00:00

Owner

Originally created by @vshortt73 on GitHub (Feb 22, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1859

Reminder

I have read the above rules and searched the existing issues.

System Info

kt-kernel: main @ 16a8b98 (support qwen3.5 #1846)
SGLang: main @ 45095bac7
Python: 3.11
GPU: RTX 5090 (32GB)
CPU: AMD Ryzen 9 9900X (Zen 5, AVX-512 BF16)
OS: Ubuntu 24.04

Reproduction

Repository: kvcache-ai/ktransformers

Summary

Latest SGLang main (45095bac7) passes num_gpu_experts (int) to KTMoEWrapper.__new__(), but latest kt-kernel main (16a8b98) expects gpu_experts_mask (torch.Tensor). This causes a TypeError on any model using KTransformers expert offloading.

Error

File "/programs/kt-sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 219, in create_weights
    self.wrapper = KTMoEWrapper(
TypeError: KTMoEWrapper.__new__() got an unexpected keyword argument 'num_gpu_experts'

Root Cause

SGLang's kt_ep_wrapper.py line 225:

self.wrapper = KTMoEWrapper(
    ...
    num_gpu_experts=self.num_gpu_experts,  # passes int
    ...
)

But kt_kernel.KTMoEWrapper.__new__() signature expects:

(cls, layer_idx: int, num_experts: int, ..., gpu_experts_mask: Optional[torch.Tensor], ...)

Others

Workaround

Patch kt_ep_wrapper.py to convert the int to a boolean mask:

# Before the KTMoEWrapper() call, add:
gpu_experts_mask = torch.zeros(num_experts, dtype=torch.bool)
gpu_experts_mask[:self.num_gpu_experts] = True

# Then replace:
#   num_gpu_experts=self.num_gpu_experts,
# with:
#   gpu_experts_mask=gpu_experts_mask,

Suggestion

Either:

Update kt-kernel to accept num_gpu_experts as an alternative parameter, or
Update SGLang's wrapper to construct the mask, or
Document the expected API contract between the two projects

This affects any model using KTransformers offloading, not just gpt-oss.

Originally created by @vshortt73 on GitHub (Feb 22, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1859 ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info - **kt-kernel**: `main` @ 16a8b98 (support qwen3.5 #1846) - **SGLang**: `main` @ 45095bac7 - **Python**: 3.11 - **GPU**: RTX 5090 (32GB) - **CPU**: AMD Ryzen 9 9900X (Zen 5, AVX-512 BF16) - **OS**: Ubuntu 24.04 ### Reproduction ## Repository: kvcache-ai/ktransformers ## Summary Latest SGLang `main` (45095bac7) passes `num_gpu_experts` (int) to `KTMoEWrapper.__new__()`, but latest kt-kernel `main` (16a8b98) expects `gpu_experts_mask` (torch.Tensor). This causes a `TypeError` on any model using KTransformers expert offloading. ## Error ``` File "/programs/kt-sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 219, in create_weights self.wrapper = KTMoEWrapper( TypeError: KTMoEWrapper.__new__() got an unexpected keyword argument 'num_gpu_experts' ``` ## Root Cause SGLang's `kt_ep_wrapper.py` line 225: ```python self.wrapper = KTMoEWrapper( ... num_gpu_experts=self.num_gpu_experts, # passes int ... ) ``` But `kt_kernel.KTMoEWrapper.__new__()` signature expects: ```python (cls, layer_idx: int, num_experts: int, ..., gpu_experts_mask: Optional[torch.Tensor], ...) ``` ### Others ## Workaround Patch `kt_ep_wrapper.py` to convert the int to a boolean mask: ```python # Before the KTMoEWrapper() call, add: gpu_experts_mask = torch.zeros(num_experts, dtype=torch.bool) gpu_experts_mask[:self.num_gpu_experts] = True # Then replace: # num_gpu_experts=self.num_gpu_experts, # with: # gpu_experts_mask=gpu_experts_mask, ``` ## Suggestion Either: 1. Update kt-kernel to accept `num_gpu_experts` as an alternative parameter, or 2. Update SGLang's wrapper to construct the mask, or 3. Document the expected API contract between the two projects This affects any model using KTransformers offloading, not just gpt-oss.

bsakaguchi closed this issue

2026-10-29 14:49:22 +00:00

bsakaguchi commented

2026-10-29 14:49:22 +00:00

Author

Owner

@ErvinXie commented on GitHub (Feb 28, 2026):

This API mismatch occurs because you are using SGLang upstream main branch, which is not synchronized with kt-kernel's interface. Please use the KTransformers fork of SGLang (https://github.com/kvcache-ai/sglang) instead, where the API contract between SGLang and kt-kernel is kept in sync.

This reply was generated by an AI agent and reviewed by a human.

@ErvinXie commented on GitHub (Feb 28, 2026): This API mismatch occurs because you are using SGLang upstream `main` branch, which is not synchronized with kt-kernel's interface. Please use the **KTransformers fork of SGLang** (https://github.com/kvcache-ai/sglang) instead, where the API contract between SGLang and kt-kernel is kept in sync. --- *This reply was generated by an AI agent and reviewed by a human.*

bsakaguchi commented

2026-10-29 14:49:23 +00:00

Author

Owner

@ErvinXie commented on GitHub (Feb 28, 2026):

This is caused by using SGLang upstream main instead of the KTransformers fork. Please use https://github.com/kvcache-ai/sglang. Closing.

@ErvinXie commented on GitHub (Feb 28, 2026): This is caused by using SGLang upstream main instead of the KTransformers fork. Please use https://github.com/kvcache-ai/sglang. Closing.

bsakaguchi referenced this issue

2026-10-29 14:53:34 +00:00

[PR #1157] [MERGED] fix some bugs #1571