kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1781] Crash with 2 round prompt with kt-kernel 0.5.0.post 1 & kimi k2 thinking #1120

New issue
Closed
opened 2026-10-29 14:48:46 +00:00 by bsakaguchi · 1 comment
bsakaguchi commented 2026-10-29 14:48:46 +00:00
Owner
Copy link

Originally created by @mrgaolei on GitHub (Jan 4, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1781

Reminder

  • I have read the above rules and searched the existing issues.

System Info

CPU: 2 x 6416h
RAM: 1T DDR5
GPU: 4090 48G

Ubuntu 24.04
python 3.12
cuda 13

start up script:

python -m sglang.launch_server \
  --host 0.0.0.0 \
  --port 10002 \
  --model /opt/ai-models/Kimi-K2-Thinking \
  --kt-weight-path /opt/ai-models/Kimi-K2-Thinking \
  --kt-cpuinfer 52 \
  --kt-threadpool-count 2 \
  --kt-num-gpu-experts 0 \
  --kt-method RAWINT4 \
  --kt-gpu-prefill-token-threshold 400 \
  --kt-max-deferred-experts-per-token 1 \
  --trust-remote-code \
  --mem-fraction-static 0.94 \
  --served-model-name Kimi-K2-Thinking \
  --enable-mixed-chunk \
  --enable-p2p-check \
  --disable-shared-experts-fusion \
  --chunked-prefill-size 30000 \
  --max-total-tokens 30000 \
  --attention-backend flashinfer

Reproduction

The script starts fine, and the first question works, but when I ask a question a second time in OpenWebUI, the script crashes. This happens every time.

[rank0]:[W104 16:12:19.144829005 ProcessGroupNCCL.cpp:5023] [PG ID 2 PG GUID 3 Rank 0]  using GPU 0 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can specify device_id in init_process_group() to force use of a particular device.
Fatal Python error: Fatal Python error: Segmentation faultSegmentation fault



Thread 0x0000737529ffe6c0 (most recent call first):
  File "/home/aigao/sglang/python/sglang/srt/managers/scheduler_runtime_checker_mixin.py", line 250 in watchdog_thread
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1012 in run
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1032 in _bootstrap

Thread 0x00007421fbfff6c0 (most recent call first):
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 359 in wait
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 655 in wait
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/tqdm/_monitor.py", line 60 in run
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1032 in _bootstrap

Thread 0x000074221ffff6c0 (most recent call first):
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 359 in wait
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 655 in wait
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/tqdm/_monitor.py", line 60 in run
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1032 in _bootstrap

Thread 0x000074b61687b6c0 (most recent call first):
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/_inductor/compile_worker/subproc_pool.py", line 61 in _recv_msg
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/_inductor/compile_worker/subproc_pool.py", line 195 in _read_thread
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1012 in run
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1032 in _bootstrap

Thread 0x000074ba55960740 (most recent call first):
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/kt_kernel/utils/amx.py", line 512 in sync_write_weight_scale_to_buffer
  File "/home/aigao/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 360 in load
  File "/home/aigao/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 744 in _build_full_context
  File "/home/aigao/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 671 in apply
  File "/home/aigao/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 867 in run_moe_core
  File
Extension modules: "numpy._core._multiarray_umath/home/aigao/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 845 in forward
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1784 in _call_impl
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", , line numpy.linalg._umath_linalg1773 in _wrapped_call_impl
  File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", line 837 in forward_normal
  File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", line 756 in forward
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1784 in _call_impl
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1773 in _wrapped_call_impl
  File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", line 2716 in forward,
pybase64._pybase64  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1784 in _call_impl
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1773 in _wrapped_call_impl
  File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", , line 2965charset_normalizer.md in forward
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", , line requests.packages.charset_normalizer.md1784 in _call_impl
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py, "requests.packages.chardet.md, line 1773 in _wrapped_call_impl
  File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", line 3104 in forward
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120 in decorate_context
  File "/home/aigao/sglang/python/sglang/srt/model_executor/model_runner.py", line 2100 in forward_extend
  File "/home/aigao/sglang/python/sglang/srt/model_executor/model_runner.py", line 2212 in _forward_raw
  File "/home/aigao/sglang/python/sglang/srt/model_executor/model_runner.py", line 2155 in forward
  File "/home/aigao/sglang/python/sglang/srt/managers/tp_worker.py", line 371 in forward_batch_generation
,   File multidict._multidict"/home/aigao/sglang/python/sglang/srt/managers/scheduler.py", line 1995 in run_batch
  File ", /home/aigao/sglang/python/sglang/srt/managers/scheduler.pyyarl._quoting_c", line 993 in event_loop_overlap
  File , "propcache._helpers_c/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120 in decorate_context
  File , "aiohttp._http_writer/home/aigao/sglang/python/sglang/srt/managers/scheduler.py", line 2698 in , run_scheduler_processaiohttp._http_parser
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/multiprocessing/process.py, "aiohttp._websocket.mask, line 108 in run
  File , "aiohttp._websocket.reader_c/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/multiprocessing/process.py", line 314 in _bootstrap
  File , "frozenlist._frozenlist/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/multiprocessing/spawn.py", line 135 in _main
  File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/multiprocessing/spawn.py, "torch._C, line 122 in spawn_main
  File ", <string>torch._C._dynamo.autograd_compiler", line 1 in <module>, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special
Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, pybase64._pybase64, charset_normalizer.md, requests.packages.charset_normalizer.md, , requests.packages.chardet.mdpsutil._psutil_linux, zmq.backend.cython._zmq, multidict._multidict, , yarl._quotExtension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, pybase64._pybase64, charset_normalizer.md, requests.packages.charset_normalizer.md, , requests.packages.chardet.mdpsutil._psutil_linux, zmq.backend.cython._zmq, multidict._multidict, , yarl._quoting_cPIL._imaging, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, , sentencepiece._sentencepiecefrozenlist._frozenlist, , torch._Cyaml._yaml, torch._C._dynamo.autograd_compiler, , torch._C._dynamo.evaing_cPIL._imaging, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, , sentencepiece._sentencepiecefrozenlist._frozenlist, , torch._Cyaml._yaml, torch._C._dynamo.autograd_compiler, , torch._C._dynamo.eval_frameregex._regex, torch._C._dynamo.guards, , torch._C._dynamo.utilsmarkupsafe._speedups, torch._C._fft, torch._C._linalg, torch._C._nested, , PIL._imagingfttorch._C._nn, torch._C._sparse, torch._C._special, , numpy.random._commonpsutil._psutil_linux, numpy.random.bit_gel_frameregex._regex, torch._C._dynamo.guards, , torch._C._dynamo.utilsmarkupsafe._speedups, torch._C._fft, torch._C._linalg, torch._C._nested, , PIL._imagingfttorch._C._nn, torch._C._sparse, torch._C._special, , numpy.random._commonpsutil._psutil_linux, numpy.random.bit_generator, zmq.backend.cython._zmq, numpy.random._bounded_integers, PIL._imaging, numpy.random._pcg64, numpy.random._generator, , sentencepiece._sentencepiecenumpy.random._mt19937, yaml._yaml, numpy.random._philox, regex._regex, numpy.random._sfc64, markupsafe._speedups, numnerator, zmq.backend.cython._zmq, numpy.random._bounded_integers, PIL._imaging, numpy.random._pcg64, numpy.random._generator, , sentencepiece._sentencepiecenumpy.random._mt19937, yaml._yaml, numpy.random._philox, regex._regex, numpy.random._sfc64, markupsafe._speedups, numpy.random.mtrand, PIL._imagingft, _cffi_backend, scipy._lib._ccallback_c, , numpy.random._commonscipy.linalg._fblas, , numpy.random.bit_generatorscipy.linalg._flapack, numpy.random._bounded_integers, _cyutility, numpy.random._pcg64, scipy._cyutility, numpy.random._generatopy.random.mtrand, PIL._imagingft, _cffi_backend, scipy._lib._ccallback_c, , numpy.random._commonscipy.linalg._fblas, , numpy.random.bit_generatorscipy.linalg._flapack, numpy.random._bounded_integers, _cyutility, numpy.random._pcg64, scipy._cyutility, numpy.random._generator, scipy.linalg.cython_lapack, numpy.random._mt19937, , numpy.random._philoxscipy.linalg._cythonized_array_utils, numpy.random._sfc64, , scipy.linalg._solve_toeplitznumpy.random.mtrand, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_schur_sqrtm, , _cffi_backendscipr, scipy.linalg.cython_lapack, numpy.random._mt19937, , numpy.random._philoxscipy.linalg._cythonized_array_utils, numpy.random._sfc64, , scipy.linalg._solve_toeplitznumpy.random.mtrand, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_schur_sqrtm, , _cffi_backendscipy.linalg._matfuncs_expm, , scipy.linalg._linalg_pythranscipy._lib._ccallback_c, scipy.linalg.cython_blas, scipy.linalg._fblas, scipy.linalg._decomp_update, scipy.linalg._flapack, scipy.sparse._sparsetools, _cyutility, _csparsetools, scipy._cyutility, scipy.sparse._csparsety.linalg._matfuncs_expm, , scipy.linalg._linalg_pythranscipy._lib._ccallback_c, scipy.linalg.cython_blas, scipy.linalg._fblas, scipy.linalg._decomp_update, scipy.linalg._flapack, scipy.sparse._sparsetools, _cyutility, _csparsetools, scipy._cyutility, scipy.sparse._csparsetools, scipy.linalg.cython_lapack, scipy.sparse.linalg._dsolve._superlu, , scipy.linalg._cythonized_array_utilsscipy.sparse.linalg._eigen.arpack._arpack, scipy.linalg._solve_toeplitz, scipy.sparse.linalg._propack._spropack, scipy.linalg._decomp_lu_cython, scipy.sparse.linalools, scipy.linalg.cython_lapack, scipy.sparse.linalg._dsolve._superlu, , scipy.linalg._cythonized_array_utilsscipy.sparse.linalg._eigen.arpack._arpack, scipy.linalg._solve_toeplitz, scipy.sparse.linalg._propack._spropack, scipy.linalg._decomp_lu_cython, scipy.sparse.linalg._propack._dpropack, scipy.linalg._matfuncs_schur_sqrtm, scipy.sparse.linalg._propack._cpropack, scipy.linalg._matfuncs_expm, scipy.sparse.linalg._propack._zpropack, scipy.linalg._linalg_pythran, scipy.optimize._group_columns, scipy.linalg.cython_blas, scipy._lib.messagesg._propack._dpropack, scipy.linalg._matfuncs_schur_sqrtm, scipy.sparse.linalg._propack._cpropack, scipy.linalg._matfuncs_expm, scipy.sparse.linalg._propack._zpropack, scipy.linalg._linalg_pythran, scipy.optimize._group_columns, scipy.linalg.cython_blas, scipy._lib.messagestream, scipy.linalg._decomp_update, scipy.optimize._trlib._trlib, scipy.sparse._sparsetools, scipy.optimize._lbfgsb, _csparsetools, , _moduleTNCscipy.sparse._csparsetools, scipy.optimize._moduleTNC, scipy.sparse.linalg._dsolve._superlu, scipy.optimize._slsqplib, scipy.spartream, scipy.linalg._decomp_update, scipy.optimize._trlib._trlib, scipy.sparse._sparsetools, scipy.optimize._lbfgsb, _csparsetools, , _moduleTNCscipy.sparse._csparsetools, scipy.optimize._moduleTNC, scipy.sparse.linalg._dsolve._superlu, scipy.optimize._slsqplib, scipy.sparse.linalg._eigen.arpack._arpack, scipy.optimize._minpack, scipy.sparse.linalg._propack._spropack, scipy.optimize._lsq.givens_elimination, scipy.sparse.linalg._propack._dpropack, , scipy.optimize._zerosscipy.sparse.linalg._propack._cpropack, , scipy._lib._uarray._uarrayscipse.linalg._eigen.arpack._arpack, scipy.optimize._minpack, scipy.sparse.linalg._propack._spropack, scipy.optimize._lsq.givens_elimination, scipy.sparse.linalg._propack._dpropack, , scipy.optimize._zerosscipy.sparse.linalg._propack._cpropack, , scipy._lib._uarray._uarrayscipy.sparse.linalg._propack._zpropack, , scipy.special._ufuncs_cxxscipy.optimize._group_columns, , scipy.special._ellip_harm_2scipy._lib.messagestream, scipy.special._special_ufuncs, scipy.optimize._trlib._trlib, scipy.special._gufuncs, scipy.optimize._lbfgsb, scipy.special._y.sparse.linalg._propack._zpropack, , scipy.special._ufuncs_cxxscipy.optimize._group_columns, , scipy.special._ellip_harm_2scipy._lib.messagestream, scipy.special._special_ufuncs, scipy.optimize._trlib._trlib, scipy.special._gufuncs, scipy.optimize._lbfgsb, scipy.special._ufuncs, , _moduleTNCscipy.special._specfun, scipy.optimize._moduleTNC, scipy.special._comb, scipy.optimize._slsqplib, , scipy.linalg._decomp_interpolativescipy.optimize._minpack, , scipy.optimize._bglu_densescipy.optimize._lsq.givens_elimination, scipy.optimize._lsap, scipufuncs, , _moduleTNCscipy.special._specfun, scipy.optimize._moduleTNC, scipy.special._comb, scipy.optimize._slsqplib, , scipy.linalg._decomp_interpolativescipy.optimize._minpack, , scipy.optimize._bglu_densescipy.optimize._lsq.givens_elimination, scipy.optimize._lsap, scipy.optimize._zeros, scipy.spatial._ckdtree, scipy._lib._uarray._uarray, scipy.spatial._qhull, , scipy.special._ufuncs_cxxscipy.spatial._voronoi, , scipy.special._ellip_harm_2scipy.spatial._hausdorff, scipy.special._special_ufuncs, scipy.spatial._distance_wrap, scipy.specialy.optimize._zeros, scipy.spatial._ckdtree, scipy._lib._uarray._uarray, scipy.spatial._qhull, , scipy.special._ufuncs_cxxscipy.spatial._voronoi, , scipy.special._ellip_harm_2scipy.spatial._hausdorff, scipy.special._special_ufuncs, scipy.spatial._distance_wrap, scipy.special._gufuncs, , scipy.spatial.transform._rotationscipy.special._ufuncs, , scipy.spatial.transform._rigid_transformscipy.special._specfun, , scipy.optimize._directscipy.special._comb, scipy.linalg._decomp_interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, , setpr._gufuncs, , scipy.spatial.transform._rotationscipy.special._ufuncs, , scipy.spatial.transform._rigid_transformscipy.special._specfun, , scipy.optimize._directscipy.special._comb, scipy.linalg._decomp_interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, , setproctitle._setproctitlescipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._hausdorff, scipy.spatial._distance_wrap, scipy.spatial.transform._rotation, scipy.spatial.transform._rigid_transform, cuda.bindings._bindings.cydriver, scipy.optimize.octitle._setproctitlescipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._hausdorff, scipy.spatial._distance_wrap, scipy.spatial.transform._rotation, scipy.spatial.transform._rigid_transform, cuda.bindings._bindings.cydriver, scipy.optimize._direct, cuda.bindings.cydriver, cuda.bindings.driver, tvm_ffi.core, setproctitle._setproctitle, msgspec._core, cuda.bindings._bindings.cydriver, cuda.bindings._bindings.cynvrtc, cuda.bindings.cydriver, cuda.bindings.cynvrtc, cuda.bindings.driver, cuda.bindings.nvrtc, tvm__direct, cuda.bindings.cydriver, cuda.bindings.driver, tvm_ffi.core, setproctitle._setproctitle, msgspec._core, cuda.bindings._bindings.cydriver, cuda.bindings._bindings.cynvrtc, cuda.bindings.cydriver, cuda.bindings.cynvrtc, cuda.bindings.driver, cuda.bindings.nvrtc, tvm_ffi.core, cuda.bindings._bindings.cyruntime_ptds, , msgspec._corecuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda_utils, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptffi.core, cuda.bindings._bindings.cyruntime_ptds, , msgspec._corecuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda_utils, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, , cuda.bindings.runtime__triton_launcher (total: 110)
, cuda_utils, __triton_launcher (total: 110)
!!!!!!! Segfault encountered !!!!!!!
  File "./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c", line 0, in 0x000074ba5564532f
  File "../../../../../libstdc++-v3/src/c++11/thread.cc", line 104, in execute_native_thread_routine
  File "./nptl/pthread_create.c", line 447, in start_thread
  File "../sysdeps/unix/sysv/linux/x86_64/clone3.S", line 78, in clone3
  File "<unknown>", line 0, in 0xffffffffffffffff

Others

It seem's GPU ID wrong, but with nvidia-smi every thing is OK

Sun Jan  4 16:20:31 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        Off |   00000000:2A:00.0 Off |                  Off |
| 59%   22C    P8             20W /  450W |     513MiB /  49140MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            6135      C   /usr/local/bin/python3                  490MiB |
|    0   N/A  N/A           25124      G   /usr/lib/xorg/Xorg                        4MiB |
+-----------------------------------------------------------------------------------------+
Originally created by @mrgaolei on GitHub (Jan 4, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1781 ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info CPU: 2 x 6416h RAM: 1T DDR5 GPU: 4090 48G Ubuntu 24.04 python 3.12 cuda 13 start up script: ```bash python -m sglang.launch_server \ --host 0.0.0.0 \ --port 10002 \ --model /opt/ai-models/Kimi-K2-Thinking \ --kt-weight-path /opt/ai-models/Kimi-K2-Thinking \ --kt-cpuinfer 52 \ --kt-threadpool-count 2 \ --kt-num-gpu-experts 0 \ --kt-method RAWINT4 \ --kt-gpu-prefill-token-threshold 400 \ --kt-max-deferred-experts-per-token 1 \ --trust-remote-code \ --mem-fraction-static 0.94 \ --served-model-name Kimi-K2-Thinking \ --enable-mixed-chunk \ --enable-p2p-check \ --disable-shared-experts-fusion \ --chunked-prefill-size 30000 \ --max-total-tokens 30000 \ --attention-backend flashinfer ``` ### Reproduction The script starts fine, and the first question works, but when I ask a question a second time in OpenWebUI, the script crashes. This happens every time. ```bash [rank0]:[W104 16:12:19.144829005 ProcessGroupNCCL.cpp:5023] [PG ID 2 PG GUID 3 Rank 0] using GPU 0 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can specify device_id in init_process_group() to force use of a particular device. Fatal Python error: Fatal Python error: Segmentation faultSegmentation fault Thread 0x0000737529ffe6c0 (most recent call first): File "/home/aigao/sglang/python/sglang/srt/managers/scheduler_runtime_checker_mixin.py", line 250 in watchdog_thread File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1012 in run File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1075 in _bootstrap_inner File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1032 in _bootstrap Thread 0x00007421fbfff6c0 (most recent call first): File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 359 in wait File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 655 in wait File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/tqdm/_monitor.py", line 60 in run File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1075 in _bootstrap_inner File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1032 in _bootstrap Thread 0x000074221ffff6c0 (most recent call first): File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 359 in wait File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 655 in wait File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/tqdm/_monitor.py", line 60 in run File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1075 in _bootstrap_inner File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1032 in _bootstrap Thread 0x000074b61687b6c0 (most recent call first): File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/_inductor/compile_worker/subproc_pool.py", line 61 in _recv_msg File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/_inductor/compile_worker/subproc_pool.py", line 195 in _read_thread File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1012 in run File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1075 in _bootstrap_inner File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/threading.py", line 1032 in _bootstrap Thread 0x000074ba55960740 (most recent call first): File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/kt_kernel/utils/amx.py", line 512 in sync_write_weight_scale_to_buffer File "/home/aigao/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 360 in load File "/home/aigao/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 744 in _build_full_context File "/home/aigao/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 671 in apply File "/home/aigao/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 867 in run_moe_core File Extension modules: "numpy._core._multiarray_umath/home/aigao/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 845 in forward File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1784 in _call_impl File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", , line numpy.linalg._umath_linalg1773 in _wrapped_call_impl File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", line 837 in forward_normal File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", line 756 in forward File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1784 in _call_impl File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1773 in _wrapped_call_impl File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", line 2716 in forward, pybase64._pybase64 File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1784 in _call_impl File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1773 in _wrapped_call_impl File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", , line 2965charset_normalizer.md in forward File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py", , line requests.packages.charset_normalizer.md1784 in _call_impl File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/nn/modules/module.py, "requests.packages.chardet.md, line 1773 in _wrapped_call_impl File "/home/aigao/sglang/python/sglang/srt/models/deepseek_v2.py", line 3104 in forward File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120 in decorate_context File "/home/aigao/sglang/python/sglang/srt/model_executor/model_runner.py", line 2100 in forward_extend File "/home/aigao/sglang/python/sglang/srt/model_executor/model_runner.py", line 2212 in _forward_raw File "/home/aigao/sglang/python/sglang/srt/model_executor/model_runner.py", line 2155 in forward File "/home/aigao/sglang/python/sglang/srt/managers/tp_worker.py", line 371 in forward_batch_generation , File multidict._multidict"/home/aigao/sglang/python/sglang/srt/managers/scheduler.py", line 1995 in run_batch File ", /home/aigao/sglang/python/sglang/srt/managers/scheduler.pyyarl._quoting_c", line 993 in event_loop_overlap File , "propcache._helpers_c/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120 in decorate_context File , "aiohttp._http_writer/home/aigao/sglang/python/sglang/srt/managers/scheduler.py", line 2698 in , run_scheduler_processaiohttp._http_parser File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/multiprocessing/process.py, "aiohttp._websocket.mask, line 108 in run File , "aiohttp._websocket.reader_c/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/multiprocessing/process.py", line 314 in _bootstrap File , "frozenlist._frozenlist/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/multiprocessing/spawn.py", line 135 in _main File "/home/aigao/anaconda3/envs/kt-kernel/lib/python3.12/multiprocessing/spawn.py, "torch._C, line 122 in spawn_main File ", <string>torch._C._dynamo.autograd_compiler", line 1 in <module>, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, pybase64._pybase64, charset_normalizer.md, requests.packages.charset_normalizer.md, , requests.packages.chardet.mdpsutil._psutil_linux, zmq.backend.cython._zmq, multidict._multidict, , yarl._quotExtension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, pybase64._pybase64, charset_normalizer.md, requests.packages.charset_normalizer.md, , requests.packages.chardet.mdpsutil._psutil_linux, zmq.backend.cython._zmq, multidict._multidict, , yarl._quoting_cPIL._imaging, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, , sentencepiece._sentencepiecefrozenlist._frozenlist, , torch._Cyaml._yaml, torch._C._dynamo.autograd_compiler, , torch._C._dynamo.evaing_cPIL._imaging, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, , sentencepiece._sentencepiecefrozenlist._frozenlist, , torch._Cyaml._yaml, torch._C._dynamo.autograd_compiler, , torch._C._dynamo.eval_frameregex._regex, torch._C._dynamo.guards, , torch._C._dynamo.utilsmarkupsafe._speedups, torch._C._fft, torch._C._linalg, torch._C._nested, , PIL._imagingfttorch._C._nn, torch._C._sparse, torch._C._special, , numpy.random._commonpsutil._psutil_linux, numpy.random.bit_gel_frameregex._regex, torch._C._dynamo.guards, , torch._C._dynamo.utilsmarkupsafe._speedups, torch._C._fft, torch._C._linalg, torch._C._nested, , PIL._imagingfttorch._C._nn, torch._C._sparse, torch._C._special, , numpy.random._commonpsutil._psutil_linux, numpy.random.bit_generator, zmq.backend.cython._zmq, numpy.random._bounded_integers, PIL._imaging, numpy.random._pcg64, numpy.random._generator, , sentencepiece._sentencepiecenumpy.random._mt19937, yaml._yaml, numpy.random._philox, regex._regex, numpy.random._sfc64, markupsafe._speedups, numnerator, zmq.backend.cython._zmq, numpy.random._bounded_integers, PIL._imaging, numpy.random._pcg64, numpy.random._generator, , sentencepiece._sentencepiecenumpy.random._mt19937, yaml._yaml, numpy.random._philox, regex._regex, numpy.random._sfc64, markupsafe._speedups, numpy.random.mtrand, PIL._imagingft, _cffi_backend, scipy._lib._ccallback_c, , numpy.random._commonscipy.linalg._fblas, , numpy.random.bit_generatorscipy.linalg._flapack, numpy.random._bounded_integers, _cyutility, numpy.random._pcg64, scipy._cyutility, numpy.random._generatopy.random.mtrand, PIL._imagingft, _cffi_backend, scipy._lib._ccallback_c, , numpy.random._commonscipy.linalg._fblas, , numpy.random.bit_generatorscipy.linalg._flapack, numpy.random._bounded_integers, _cyutility, numpy.random._pcg64, scipy._cyutility, numpy.random._generator, scipy.linalg.cython_lapack, numpy.random._mt19937, , numpy.random._philoxscipy.linalg._cythonized_array_utils, numpy.random._sfc64, , scipy.linalg._solve_toeplitznumpy.random.mtrand, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_schur_sqrtm, , _cffi_backendscipr, scipy.linalg.cython_lapack, numpy.random._mt19937, , numpy.random._philoxscipy.linalg._cythonized_array_utils, numpy.random._sfc64, , scipy.linalg._solve_toeplitznumpy.random.mtrand, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_schur_sqrtm, , _cffi_backendscipy.linalg._matfuncs_expm, , scipy.linalg._linalg_pythranscipy._lib._ccallback_c, scipy.linalg.cython_blas, scipy.linalg._fblas, scipy.linalg._decomp_update, scipy.linalg._flapack, scipy.sparse._sparsetools, _cyutility, _csparsetools, scipy._cyutility, scipy.sparse._csparsety.linalg._matfuncs_expm, , scipy.linalg._linalg_pythranscipy._lib._ccallback_c, scipy.linalg.cython_blas, scipy.linalg._fblas, scipy.linalg._decomp_update, scipy.linalg._flapack, scipy.sparse._sparsetools, _cyutility, _csparsetools, scipy._cyutility, scipy.sparse._csparsetools, scipy.linalg.cython_lapack, scipy.sparse.linalg._dsolve._superlu, , scipy.linalg._cythonized_array_utilsscipy.sparse.linalg._eigen.arpack._arpack, scipy.linalg._solve_toeplitz, scipy.sparse.linalg._propack._spropack, scipy.linalg._decomp_lu_cython, scipy.sparse.linalools, scipy.linalg.cython_lapack, scipy.sparse.linalg._dsolve._superlu, , scipy.linalg._cythonized_array_utilsscipy.sparse.linalg._eigen.arpack._arpack, scipy.linalg._solve_toeplitz, scipy.sparse.linalg._propack._spropack, scipy.linalg._decomp_lu_cython, scipy.sparse.linalg._propack._dpropack, scipy.linalg._matfuncs_schur_sqrtm, scipy.sparse.linalg._propack._cpropack, scipy.linalg._matfuncs_expm, scipy.sparse.linalg._propack._zpropack, scipy.linalg._linalg_pythran, scipy.optimize._group_columns, scipy.linalg.cython_blas, scipy._lib.messagesg._propack._dpropack, scipy.linalg._matfuncs_schur_sqrtm, scipy.sparse.linalg._propack._cpropack, scipy.linalg._matfuncs_expm, scipy.sparse.linalg._propack._zpropack, scipy.linalg._linalg_pythran, scipy.optimize._group_columns, scipy.linalg.cython_blas, scipy._lib.messagestream, scipy.linalg._decomp_update, scipy.optimize._trlib._trlib, scipy.sparse._sparsetools, scipy.optimize._lbfgsb, _csparsetools, , _moduleTNCscipy.sparse._csparsetools, scipy.optimize._moduleTNC, scipy.sparse.linalg._dsolve._superlu, scipy.optimize._slsqplib, scipy.spartream, scipy.linalg._decomp_update, scipy.optimize._trlib._trlib, scipy.sparse._sparsetools, scipy.optimize._lbfgsb, _csparsetools, , _moduleTNCscipy.sparse._csparsetools, scipy.optimize._moduleTNC, scipy.sparse.linalg._dsolve._superlu, scipy.optimize._slsqplib, scipy.sparse.linalg._eigen.arpack._arpack, scipy.optimize._minpack, scipy.sparse.linalg._propack._spropack, scipy.optimize._lsq.givens_elimination, scipy.sparse.linalg._propack._dpropack, , scipy.optimize._zerosscipy.sparse.linalg._propack._cpropack, , scipy._lib._uarray._uarrayscipse.linalg._eigen.arpack._arpack, scipy.optimize._minpack, scipy.sparse.linalg._propack._spropack, scipy.optimize._lsq.givens_elimination, scipy.sparse.linalg._propack._dpropack, , scipy.optimize._zerosscipy.sparse.linalg._propack._cpropack, , scipy._lib._uarray._uarrayscipy.sparse.linalg._propack._zpropack, , scipy.special._ufuncs_cxxscipy.optimize._group_columns, , scipy.special._ellip_harm_2scipy._lib.messagestream, scipy.special._special_ufuncs, scipy.optimize._trlib._trlib, scipy.special._gufuncs, scipy.optimize._lbfgsb, scipy.special._y.sparse.linalg._propack._zpropack, , scipy.special._ufuncs_cxxscipy.optimize._group_columns, , scipy.special._ellip_harm_2scipy._lib.messagestream, scipy.special._special_ufuncs, scipy.optimize._trlib._trlib, scipy.special._gufuncs, scipy.optimize._lbfgsb, scipy.special._ufuncs, , _moduleTNCscipy.special._specfun, scipy.optimize._moduleTNC, scipy.special._comb, scipy.optimize._slsqplib, , scipy.linalg._decomp_interpolativescipy.optimize._minpack, , scipy.optimize._bglu_densescipy.optimize._lsq.givens_elimination, scipy.optimize._lsap, scipufuncs, , _moduleTNCscipy.special._specfun, scipy.optimize._moduleTNC, scipy.special._comb, scipy.optimize._slsqplib, , scipy.linalg._decomp_interpolativescipy.optimize._minpack, , scipy.optimize._bglu_densescipy.optimize._lsq.givens_elimination, scipy.optimize._lsap, scipy.optimize._zeros, scipy.spatial._ckdtree, scipy._lib._uarray._uarray, scipy.spatial._qhull, , scipy.special._ufuncs_cxxscipy.spatial._voronoi, , scipy.special._ellip_harm_2scipy.spatial._hausdorff, scipy.special._special_ufuncs, scipy.spatial._distance_wrap, scipy.specialy.optimize._zeros, scipy.spatial._ckdtree, scipy._lib._uarray._uarray, scipy.spatial._qhull, , scipy.special._ufuncs_cxxscipy.spatial._voronoi, , scipy.special._ellip_harm_2scipy.spatial._hausdorff, scipy.special._special_ufuncs, scipy.spatial._distance_wrap, scipy.special._gufuncs, , scipy.spatial.transform._rotationscipy.special._ufuncs, , scipy.spatial.transform._rigid_transformscipy.special._specfun, , scipy.optimize._directscipy.special._comb, scipy.linalg._decomp_interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, , setpr._gufuncs, , scipy.spatial.transform._rotationscipy.special._ufuncs, , scipy.spatial.transform._rigid_transformscipy.special._specfun, , scipy.optimize._directscipy.special._comb, scipy.linalg._decomp_interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, , setproctitle._setproctitlescipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._hausdorff, scipy.spatial._distance_wrap, scipy.spatial.transform._rotation, scipy.spatial.transform._rigid_transform, cuda.bindings._bindings.cydriver, scipy.optimize.octitle._setproctitlescipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._hausdorff, scipy.spatial._distance_wrap, scipy.spatial.transform._rotation, scipy.spatial.transform._rigid_transform, cuda.bindings._bindings.cydriver, scipy.optimize._direct, cuda.bindings.cydriver, cuda.bindings.driver, tvm_ffi.core, setproctitle._setproctitle, msgspec._core, cuda.bindings._bindings.cydriver, cuda.bindings._bindings.cynvrtc, cuda.bindings.cydriver, cuda.bindings.cynvrtc, cuda.bindings.driver, cuda.bindings.nvrtc, tvm__direct, cuda.bindings.cydriver, cuda.bindings.driver, tvm_ffi.core, setproctitle._setproctitle, msgspec._core, cuda.bindings._bindings.cydriver, cuda.bindings._bindings.cynvrtc, cuda.bindings.cydriver, cuda.bindings.cynvrtc, cuda.bindings.driver, cuda.bindings.nvrtc, tvm_ffi.core, cuda.bindings._bindings.cyruntime_ptds, , msgspec._corecuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda_utils, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptffi.core, cuda.bindings._bindings.cyruntime_ptds, , msgspec._corecuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda_utils, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, , cuda.bindings.runtime__triton_launcher (total: 110) , cuda_utils, __triton_launcher (total: 110) !!!!!!! Segfault encountered !!!!!!! File "./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c", line 0, in 0x000074ba5564532f File "../../../../../libstdc++-v3/src/c++11/thread.cc", line 104, in execute_native_thread_routine File "./nptl/pthread_create.c", line 447, in start_thread File "../sysdeps/unix/sysv/linux/x86_64/clone3.S", line 78, in clone3 File "<unknown>", line 0, in 0xffffffffffffffff ``` ### Others It seem's GPU ID wrong, but with `nvidia-smi` every thing is OK ```bash Sun Jan 4 16:20:31 2026 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 580.95.05 Driver Version: 580.95.05 CUDA Version: 13.0 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 4090 Off | 00000000:2A:00.0 Off | Off | | 59% 22C P8 20W / 450W | 513MiB / 49140MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 6135 C /usr/local/bin/python3 490MiB | | 0 N/A N/A 25124 G /usr/lib/xorg/Xorg 4MiB | +-----------------------------------------------------------------------------------------+ ```
bsakaguchi commented 2026-10-29 14:48:47 +00:00
Author
Owner
Copy link

@mrgaolei commented on GitHub (Jan 6, 2026):

I reinstall carefully it fixed.

<!-- gh-comment-id:3712930098 --> @mrgaolei commented on GitHub (Jan 6, 2026): I reinstall carefully it fixed.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#1120
No description provided.