mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1951] kt-kernel 0.6.1 requires torch<2.12,>=2.10, but you have torch 2.9.1 which is incompatible. #1209
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#1209
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @cadslab on GitHub (Apr 25, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1951
Reminder
System Info
k1@k0:~$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 43 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 24
On-line CPU(s) list: 0-23
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7402P 24-Core Processor
CPU family: 23
Model: 49
Thread(s) per core: 1
Core(s) per socket: 24
Socket(s): 1
Stepping: 0
Frequency boost: enabled
CPU max MHz: 2800.0000
CPU min MHz: 1500.0000
BogoMIPS: 5599.51
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nons
top_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm
sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fs
gsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsavee
rptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip r
dpid overflow_recov succor smca sme sev sev_es ibpb_exit_to_user
Virtualization features:
Virtualization: AMD-V
Caches (sum of all):
L1d: 768 KiB (24 instances)
L1i: 768 KiB (24 instances)
L2: 12 MiB (24 instances)
L3: 128 MiB (8 instances)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-23
Vulnerabilities:
Gather data sampling: Not affected
Indirect target selection: Not affected
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Not affected
Reg file data sampling: Not affected
Retbleed: Mitigation; untrained return thunk; SMT disabled
Spec rstack overflow: Mitigation; SMT disabled
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Retpolines; IBPB conditional; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Srbds: Not affected
Tsa: Not affected
Tsx async abort: Not affected
Vmscape: Mitigation; IBPB before exit to userspace
k1@k0:~$ nvidia-smi
Sat Apr 25 11:45:21 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05 Driver Version: 580.95.05 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX 4000 Ada Gene... On | 00000000:04:00.0 Off | Off |
| 30% 52C P2 34W / 130W | 3356MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA RTX 4000 Ada Gene... On | 00000000:05:00.0 Off | Off |
| 30% 55C P2 35W / 130W | 3308MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA RTX 4000 Ada Gene... On | 00000000:43:00.0 Off | Off |
| 30% 48C P2 28W / 130W | 3308MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 3 NVIDIA RTX 4000 Ada Gene... On | 00000000:44:00.0 Off | Off |
| 30% 48C P2 30W / 130W | 3308MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 4 NVIDIA RTX 4000 Ada Gene... On | 00000000:88:00.0 Off | Off |
| 30% 49C P2 30W / 130W | 3308MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 5 NVIDIA RTX 4000 Ada Gene... On | 00000000:89:00.0 Off | Off |
| 30% 55C P2 33W / 130W | 3308MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 6 NVIDIA RTX 4000 Ada Gene... On | 00000000:C3:00.0 Off | Off |
| 30% 48C P2 31W / 130W | 3308MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 7 NVIDIA RTX 4000 Ada Gene... On | 00000000:C4:00.0 Off | Off |
| 30% 49C P2 33W / 130W | 3308MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
Reproduction
follow installation guide
Others
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchvision 0.24.1 requires torch==2.9.1, but you have torch 2.11.0 which is incompatible.
torchaudio 2.9.1 requires torch==2.9.1, but you have torch 2.11.0 which is incompatible.
sglang-kt 0.6.1 requires torch==2.9.1, but you have torch 2.11.0 which is incompatible.
cuda-python 12.9.0 requires cuda-bindings~=12.9.0, but you have cuda-bindings 13.2.0 which is incompatible.
Successfully installed cuda-bindings-13.2.0 kt-kernel-0.6.1 torch-2.11.0 triton-3.6.0
fixed with:
pip install sglang-kt==0.5.3
pip install torch==2.9.1
@JimmyPeilinLi commented on GitHub (Apr 25, 2026):
Thanks for reporting this. This was a packaging dependency mismatch in the
sglang-ktpackage metadata for the KT inference path, not a hardware/runtime issue.To clarify the package roles:
kt-kernelis the shared KT backend used by both KT inference and KT fine-tuning.sglang-ktis the SGLang fork/package used for the KT inference path. It is not meant to be tightly tied to the top-levelktransformerspackage.ktransformers[sft]stack, which also depends onkt-kernel, but does not requiresglang-kt.The conflict was that
kt-kernel==0.6.1expects the newer torch stack (torch>=2.10,<2.12), while the oldsglang-kt==0.6.1package metadata was still pinningtorch==2.9.1/torchaudio==2.9.1.cuda-python==12.9could also conflict with newercuda-bindings.The dependency ranges were fixed in
sglang-kthere: https://github.com/kvcache-ai/sglang/pull/32The fixed
sglang-kt==0.6.1wheel has now been published to PyPI: https://pypi.org/project/sglang-kt/0.6.1/Please retry the KT inference install path with the current
sglang-kt==0.6.1; it should no longer force torch back to 2.9.1.The workaround you listed (
sglang-kt==0.5.3+torch==2.9.1) avoids the resolver warning, but it is not the intended dependency stack forkt-kernel==0.6.1.@cadslab commented on GitHub (Apr 26, 2026):
Installation indeed has no error.
Errors occured when running:
ImportError:
[sgl_kernel] CRITICAL: Could not load any common_ops library!
Attempted locations:
GPU Info:
Please ensure sgl_kernel is properly installed with:
pip install --upgrade sgl_kernel
Error details from previous import attempts:
@cadslab commented on GitHub (Apr 26, 2026):
pip install --upgrade sgl_kernel returns
Requirement already satisfied: sgl_kernel in /home/k1/miniconda3/envs/kt/lib/python3.11/site-packages (0.3.21)
which did not solve the issue.
@JimmyPeilinLi commented on GitHub (Apr 26, 2026):
I'm very sorry. I will further handle this issue during working hours, conduct a complete verification for the 0.6.1 release, and comment the relevant information after the #1956 issue.
@JimmyPeilinLi commented on GitHub (Apr 30, 2026):
Thanks for the follow-up. We re-verified the v0.6.1 release path after the packaging fixes.
The current baseline is
torch==2.9.1with the post-release PyPI set. Please avoid the oldsglang-kt==0.6.1.post1; that file has bad metadata. The effective fixed path is:kt-kernel==0.6.1.post1sglang-kt==0.6.1.post2Related fixes: kvcache-ai/sglang#35, #1962, #1964.
If this still fails after reinstalling the latest
kt-kernel+sglang-kt, the remaining issue is likely the nativesgl_kernel/common_opsABI load, not the Python package resolver. Please share thetorch,sglang-kt,sgl-kernelversions and the fullimport sgl_kerneltraceback.@cadslab commented on GitHub (May 6, 2026):
while reinstalling sglang:
seems still not fixed
@cadslab commented on GitHub (May 7, 2026):
(ktf) k1@k0:~/workspace/ktransformers$ python -m sglang.launch_server
--host 0.0.0.0 --port 30000
--model /mnt/gpu_models/Qwen3.5-122B-A10B-FP8
--kt-weight-path /mnt/gpu_models/Qwen3.5-122B-A10B-FP8
--kt-cpuinfer 12
--kt-threadpool-count 1
--kt-num-gpu-experts 32
--kt-method FP8
--kt-gpu-prefill-token-threshold 400
--attention-backend triton
--trust-remote-code
--mem-fraction-static 0.80
--chunked-prefill-size 8192
--max-running-requests 2
--served-model-name Qwen3.5
--enable-mixed-chunk
--tensor-parallel-size 8
--disable-shared-experts-fusion
--allow-auto-truncate
--tool-call-parser qwen3_coder
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/k1/miniconda3/envs/ktf/lib/python3.12/site-packages/sglang/launch_server.py", line 30, in
from sglang.srt.server_args import prepare_server_args
File "/home/k1/miniconda3/envs/ktf/lib/python3.12/site-packages/sglang/srt/server_args.py", line 71, in
from sglang.srt.utils.hf_transformers_utils import check_gguf_file, get_config
File "/home/k1/miniconda3/envs/ktf/lib/python3.12/site-packages/sglang/srt/utils/hf_transformers_utils.py", line 78, in
from sglang.srt.configs.deepseek_v4 import DeepSeekV4Config
File "/home/k1/miniconda3/envs/ktf/lib/python3.12/site-packages/sglang/srt/configs/deepseek_v4.py", line 7, in
@dataclass
^^^^^^^^^
File "/home/k1/miniconda3/envs/ktf/lib/python3.12/dataclasses.py", line 1275, in dataclass
return wrap(cls)
^^^^^^^^^
File "/home/k1/miniconda3/envs/ktf/lib/python3.12/dataclasses.py", line 1265, in wrap
return _process_class(cls, init, repr, eq, order, unsafe_hash,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/k1/miniconda3/envs/ktf/lib/python3.12/dataclasses.py", line 1063, in _process_class
_init_fn(all_init_fields,
File "/home/k1/miniconda3/envs/ktf/lib/python3.12/dataclasses.py", line 585, in _init_fn
raise TypeError(f'non-default argument {f.name!r} '
TypeError: non-default argument 'quantization_config' follows default argument
@cadslab commented on GitHub (May 7, 2026):
got error "RuntimeError: The size of tensor a (1536) must match the size of tensor b (3072) at non-singleton dimension 1"
@cadslab commented on GitHub (May 7, 2026):
"ValueError: No experts found for key model.language_model.layers.0.mlp.experts" after switching to bf16 based on #1956
maybe just wait for sglang 0.5.11