kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1758] GLM-4.6V MOE_INT8 #1109

New issue
Open
opened 2026-10-29 14:48:39 +00:00 by bsakaguchi · 1 comment
bsakaguchi commented 2026-10-29 14:48:39 +00:00
Owner
Copy link

Originally created by @puffinjiang on GitHub (Dec 26, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1758

Originally assigned to: @ovowei on GitHub.

Reminder

  • I have read the above rules and searched the existing issues.

System Info

cpu：

Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          46 bits physical, 57 bits virtual
  Byte Order:             Little Endian
CPU(s):                   64
  On-line CPU(s) list:    0-63
Vendor ID:                AuthenticAMD
  Model name:             AMD EPYC 9354 32-Core Processor
    CPU family:           25
    Model:                17
    Thread(s) per core:   2
    Core(s) per socket:   32
    Socket(s):            1
    Stepping:             1
    Frequency boost:      enabled
    CPU max MHz:          3250.0000
    CPU min MHz:          1500.0000
    BogoMIPS:             6499.96

GPU

nvidia-smi
Fri Dec 26 21:48:22 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.105.08             Driver Version: 580.105.08     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090        On  |   00000000:01:00.0 Off |                  N/A |
| 34%   26C    P8             25W /  350W |      18MiB /  24576MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 3090        On  |   00000000:02:00.0  On |                  N/A |
| 43%   51C    P2            118W /  350W |     675MiB /  24576MiB |      9%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

dep

kt-kernel: 0.4.4
slang: 0.5.6.post2, apply the https://github.com/sgl-project/sglang/pull/14662
transformers: 5.0.0rc1

Reproduction

launch command:

python -m sglang.launch_server --host 0.0.0.0 --port 8000 --trust-remote-code --mem-fraction-static 0.85 --chunked-prefill-size 4096 --served-model-name GLM-4.6V --enable-mixed-chunk  --kt-weight-path /home/ubuntu/Documents/models/GLM-4.6V-MOE-INT8   --model-path /home/ubuntu/Documents/models/GLM-4.6V   --kt-cpuinfer 30  --kt-threadpool-count 1   --kt-num-gpu-experts 1   --kt-max-deferred-experts-per-token 1 --kt-method MOE_INT8

error info:

[2025-12-26 21:59:24] Multimodal attention backend not set. Use triton_attn.
[2025-12-26 21:59:24] Using triton_attn as multimodal attention backend.
Loading safetensors checkpoint shards:   0% Completed | 0/41 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:   7% Completed | 3/41 [00:00<00:01, 21.04it/s]
Loading safetensors checkpoint shards:  15% Completed | 6/41 [00:00<00:01, 17.61it/s]
Loading safetensors checkpoint shards:  20% Completed | 8/41 [00:00<00:01, 16.68it/s]
Loading safetensors checkpoint shards:  24% Completed | 10/41 [00:00<00:01, 15.82it/s]
Loading safetensors checkpoint shards:  29% Completed | 12/41 [00:00<00:02, 13.15it/s]
Loading safetensors checkpoint shards:  34% Completed | 14/41 [00:00<00:01, 14.71it/s]
Loading safetensors checkpoint shards:  39% Completed | 16/41 [00:01<00:01, 16.01it/s]
Loading safetensors checkpoint shards:  44% Completed | 18/41 [00:01<00:01, 15.98it/s]
Loading safetensors checkpoint shards:  49% Completed | 20/41 [00:01<00:01, 16.69it/s]
Loading safetensors checkpoint shards:  54% Completed | 22/41 [00:01<00:01, 11.38it/s]
Loading safetensors checkpoint shards:  59% Completed | 24/41 [00:01<00:01, 13.08it/s]
Loading safetensors checkpoint shards:  63% Completed | 26/41 [00:01<00:01, 14.59it/s]
Loading safetensors checkpoint shards:  68% Completed | 28/41 [00:01<00:00, 15.38it/s]
Loading safetensors checkpoint shards:  73% Completed | 30/41 [00:01<00:00, 15.96it/s]
Loading safetensors checkpoint shards:  78% Completed | 32/41 [00:02<00:00, 16.76it/s]
Loading safetensors checkpoint shards:  83% Completed | 34/41 [00:02<00:00, 17.60it/s]
Loading safetensors checkpoint shards:  88% Completed | 36/41 [00:02<00:00, 17.44it/s]
Loading safetensors checkpoint shards:  95% Completed | 39/41 [00:02<00:00, 18.39it/s]
Loading safetensors checkpoint shards: 100% Completed | 41/41 [00:02<00:00, 11.53it/s]
Loading safetensors checkpoint shards: 100% Completed | 41/41 [00:02<00:00, 14.67it/s]

TP MOE layer 1, pool: 0x47814860, expert num: 129, num_experts_per_tok: 9
  Creating AMD_MOE_TP 0 at numa 0
TP Load from loader
Fatal Python error: Segmentation fault

Thread 0x00007a992ffff640 (most recent call first):
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/threading.py", line 331 in wait
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/threading.py", line 629 in wait
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/site-packages/tqdm/_monitor.py", line 60 in run
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/threading.py", line 1002 in _bootstrap

Thread 0x00007ab9f3e20740 (most recent call first):
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel/utils/moe_kernel.py", line 306 in load_weights
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 258 in process_weights_after_loading
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_loader/loader.py", line 657 in load_weights_and_postprocess
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_loader/loader.py", line 638 in load_model
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_executor/model_runner.py", line 900 in load_model
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_executor/model_runner.py", line 432 in initialize
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_executor/model_runner.py", line 355 in __init__
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/managers/tp_worker.py", line 253 in __init__
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/managers/scheduler.py", line 471 in init_model_worker
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/managers/scheduler.py", line 333 in __init__
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/managers/scheduler.py", line 2903 in run_scheduler_process
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/process.py", line 108 in run
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/process.py", line 314 in _bootstrap
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/spawn.py", line 135 in _main
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/spawn.py", line 122 in spawn_main
  File "<string>", line 1 in <module>

Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, pybase64._pybase64, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, zmq.backend.cython._zmq, PIL._imaging, sentencepiece._sentencepiece, regex._regex, yaml._yaml, markupsafe._speedups, cuda_utils, PIL._imagingft, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._pcg64, numpy.random._generator, numpy.random._mt19937, numpy.random._philox, numpy.random._sfc64, numpy.random.mtrand, _cffi_backend, scipy._lib._ccallback_c, scipy.linalg._fblas, scipy.linalg._flapack, _cyutility, scipy._cyutility, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_schur_sqrtm, scipy.linalg._matfuncs_expm, scipy.linalg._linalg_pythran, scipy.linalg.cython_blas, scipy.linalg._decomp_update, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.optimize._group_columns, scipy._lib.messagestream, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._slsqplib, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy._lib._uarray._uarray, scipy.special._ufuncs_cxx, scipy.special._ellip_harm_2, scipy.special._special_ufuncs, scipy.special._gufuncs, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.linalg._decomp_interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._hausdorff, scipy.spatial._distance_wrap, scipy.spatial.transform._rotation, scipy.spatial.transform._rigid_transform, scipy.optimize._direct, setproctitle._setproctitle, cuda.bindings._bindings.cydriver, cuda.bindings.cydriver, cuda.bindings.driver, tvm_ffi.core, msgspec._core, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime (total: 109)
load from safetensor!!!!!!! Segfault encountered !!!!!!!
  File "./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c", line 0, in 0x00007ab9f3a4251f
  File "./nptl/pthread_kill.c", line 44, in __pthread_kill_implementation
  File "./nptl/pthread_kill.c", line 78, in __pthread_kill_internal
  File "./nptl/pthread_kill.c", line 89, in __GI___pthread_kill
  File "../sysdeps/posix/raise.c", line 26, in __GI_raise
  File "./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c", line 0, in 0x00007ab9f3a4251f
  File "../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S", line 317, in __memmove_avx512_unaligned_erms
  File "../../../../../libstdc++-v3/src/c++11/thread.cc", line 104, in execute_native_thread_routine
  File "./nptl/pthread_create.c", line 442, in start_thread
  File "../sysdeps/unix/sysv/linux/x86_64/clone3.S", line 81, in __GI___clone3
  File "<unknown>", line 0, in 0xffffffffffffffff

[2025-12-26 21:59:28] Rank 0 scheduler is dead. Please check if there are relevant logs.
[2025-12-26 21:59:28] Exit code: -11
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/ubuntu/Documents/sglang/python/sglang/launch_server.py", line 32, in <module>
    run_server(server_args)
  File "/home/ubuntu/Documents/sglang/python/sglang/launch_server.py", line 25, in run_server
    launch_server(server_args)
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/entrypoints/http_server.py", line 1692, in launch_server
    _launch_subprocesses(
  File "/home/ubuntu/Documents/sglang/python/sglang/srt/entrypoints/engine.py", line 966, in _launch_subprocesses
    data = scheduler_pipe_readers[i].recv()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
          ^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
          ^^^^^^^^^^^^^
  File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/connection.py", line 399, in _recv
    raise EOFError
EOFError

Others

No response

Originally created by @puffinjiang on GitHub (Dec 26, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1758 Originally assigned to: @ovowei on GitHub. ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info ## cpu： ``` Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 46 bits physical, 57 bits virtual Byte Order: Little Endian CPU(s): 64 On-line CPU(s) list: 0-63 Vendor ID: AuthenticAMD Model name: AMD EPYC 9354 32-Core Processor CPU family: 25 Model: 17 Thread(s) per core: 2 Core(s) per socket: 32 Socket(s): 1 Stepping: 1 Frequency boost: enabled CPU max MHz: 3250.0000 CPU min MHz: 1500.0000 BogoMIPS: 6499.96 ``` ## GPU ``` nvidia-smi Fri Dec 26 21:48:22 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 580.105.08 Driver Version: 580.105.08 CUDA Version: 13.0 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 3090 On | 00000000:01:00.0 Off | N/A | | 34% 26C P8 25W / 350W | 18MiB / 24576MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 1 NVIDIA GeForce RTX 3090 On | 00000000:02:00.0 On | N/A | | 43% 51C P2 118W / 350W | 675MiB / 24576MiB | 9% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ ``` ## dep kt-kernel: 0.4.4 slang: 0.5.6.post2, apply the https://github.com/sgl-project/sglang/pull/14662 transformers: 5.0.0rc1 ### Reproduction launch command: ```shell python -m sglang.launch_server --host 0.0.0.0 --port 8000 --trust-remote-code --mem-fraction-static 0.85 --chunked-prefill-size 4096 --served-model-name GLM-4.6V --enable-mixed-chunk --kt-weight-path /home/ubuntu/Documents/models/GLM-4.6V-MOE-INT8 --model-path /home/ubuntu/Documents/models/GLM-4.6V --kt-cpuinfer 30 --kt-threadpool-count 1 --kt-num-gpu-experts 1 --kt-max-deferred-experts-per-token 1 --kt-method MOE_INT8 ``` error info: ```text [2025-12-26 21:59:24] Multimodal attention backend not set. Use triton_attn. [2025-12-26 21:59:24] Using triton_attn as multimodal attention backend. Loading safetensors checkpoint shards: 0% Completed | 0/41 [00:00<?, ?it/s] Loading safetensors checkpoint shards: 7% Completed | 3/41 [00:00<00:01, 21.04it/s] Loading safetensors checkpoint shards: 15% Completed | 6/41 [00:00<00:01, 17.61it/s] Loading safetensors checkpoint shards: 20% Completed | 8/41 [00:00<00:01, 16.68it/s] Loading safetensors checkpoint shards: 24% Completed | 10/41 [00:00<00:01, 15.82it/s] Loading safetensors checkpoint shards: 29% Completed | 12/41 [00:00<00:02, 13.15it/s] Loading safetensors checkpoint shards: 34% Completed | 14/41 [00:00<00:01, 14.71it/s] Loading safetensors checkpoint shards: 39% Completed | 16/41 [00:01<00:01, 16.01it/s] Loading safetensors checkpoint shards: 44% Completed | 18/41 [00:01<00:01, 15.98it/s] Loading safetensors checkpoint shards: 49% Completed | 20/41 [00:01<00:01, 16.69it/s] Loading safetensors checkpoint shards: 54% Completed | 22/41 [00:01<00:01, 11.38it/s] Loading safetensors checkpoint shards: 59% Completed | 24/41 [00:01<00:01, 13.08it/s] Loading safetensors checkpoint shards: 63% Completed | 26/41 [00:01<00:01, 14.59it/s] Loading safetensors checkpoint shards: 68% Completed | 28/41 [00:01<00:00, 15.38it/s] Loading safetensors checkpoint shards: 73% Completed | 30/41 [00:01<00:00, 15.96it/s] Loading safetensors checkpoint shards: 78% Completed | 32/41 [00:02<00:00, 16.76it/s] Loading safetensors checkpoint shards: 83% Completed | 34/41 [00:02<00:00, 17.60it/s] Loading safetensors checkpoint shards: 88% Completed | 36/41 [00:02<00:00, 17.44it/s] Loading safetensors checkpoint shards: 95% Completed | 39/41 [00:02<00:00, 18.39it/s] Loading safetensors checkpoint shards: 100% Completed | 41/41 [00:02<00:00, 11.53it/s] Loading safetensors checkpoint shards: 100% Completed | 41/41 [00:02<00:00, 14.67it/s] TP MOE layer 1, pool: 0x47814860, expert num: 129, num_experts_per_tok: 9 Creating AMD_MOE_TP 0 at numa 0 TP Load from loader Fatal Python error: Segmentation fault Thread 0x00007a992ffff640 (most recent call first): File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/threading.py", line 331 in wait File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/threading.py", line 629 in wait File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/site-packages/tqdm/_monitor.py", line 60 in run File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/threading.py", line 1045 in _bootstrap_inner File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/threading.py", line 1002 in _bootstrap Thread 0x00007ab9f3e20740 (most recent call first): File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel/utils/moe_kernel.py", line 306 in load_weights File "/home/ubuntu/Documents/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 258 in process_weights_after_loading File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_loader/loader.py", line 657 in load_weights_and_postprocess File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_loader/loader.py", line 638 in load_model File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_executor/model_runner.py", line 900 in load_model File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_executor/model_runner.py", line 432 in initialize File "/home/ubuntu/Documents/sglang/python/sglang/srt/model_executor/model_runner.py", line 355 in __init__ File "/home/ubuntu/Documents/sglang/python/sglang/srt/managers/tp_worker.py", line 253 in __init__ File "/home/ubuntu/Documents/sglang/python/sglang/srt/managers/scheduler.py", line 471 in init_model_worker File "/home/ubuntu/Documents/sglang/python/sglang/srt/managers/scheduler.py", line 333 in __init__ File "/home/ubuntu/Documents/sglang/python/sglang/srt/managers/scheduler.py", line 2903 in run_scheduler_process File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/process.py", line 108 in run File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/process.py", line 314 in _bootstrap File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/spawn.py", line 135 in _main File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/spawn.py", line 122 in spawn_main File "<string>", line 1 in <module> Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, pybase64._pybase64, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, zmq.backend.cython._zmq, PIL._imaging, sentencepiece._sentencepiece, regex._regex, yaml._yaml, markupsafe._speedups, cuda_utils, PIL._imagingft, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._pcg64, numpy.random._generator, numpy.random._mt19937, numpy.random._philox, numpy.random._sfc64, numpy.random.mtrand, _cffi_backend, scipy._lib._ccallback_c, scipy.linalg._fblas, scipy.linalg._flapack, _cyutility, scipy._cyutility, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_schur_sqrtm, scipy.linalg._matfuncs_expm, scipy.linalg._linalg_pythran, scipy.linalg.cython_blas, scipy.linalg._decomp_update, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.optimize._group_columns, scipy._lib.messagestream, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._slsqplib, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy._lib._uarray._uarray, scipy.special._ufuncs_cxx, scipy.special._ellip_harm_2, scipy.special._special_ufuncs, scipy.special._gufuncs, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.linalg._decomp_interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._hausdorff, scipy.spatial._distance_wrap, scipy.spatial.transform._rotation, scipy.spatial.transform._rigid_transform, scipy.optimize._direct, setproctitle._setproctitle, cuda.bindings._bindings.cydriver, cuda.bindings.cydriver, cuda.bindings.driver, tvm_ffi.core, msgspec._core, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime (total: 109) load from safetensor!!!!!!! Segfault encountered !!!!!!! File "./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c", line 0, in 0x00007ab9f3a4251f File "./nptl/pthread_kill.c", line 44, in __pthread_kill_implementation File "./nptl/pthread_kill.c", line 78, in __pthread_kill_internal File "./nptl/pthread_kill.c", line 89, in __GI___pthread_kill File "../sysdeps/posix/raise.c", line 26, in __GI_raise File "./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c", line 0, in 0x00007ab9f3a4251f File "../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S", line 317, in __memmove_avx512_unaligned_erms File "../../../../../libstdc++-v3/src/c++11/thread.cc", line 104, in execute_native_thread_routine File "./nptl/pthread_create.c", line 442, in start_thread File "../sysdeps/unix/sysv/linux/x86_64/clone3.S", line 81, in __GI___clone3 File "<unknown>", line 0, in 0xffffffffffffffff [2025-12-26 21:59:28] Rank 0 scheduler is dead. Please check if there are relevant logs. [2025-12-26 21:59:28] Exit code: -11 Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/home/ubuntu/Documents/sglang/python/sglang/launch_server.py", line 32, in <module> run_server(server_args) File "/home/ubuntu/Documents/sglang/python/sglang/launch_server.py", line 25, in run_server launch_server(server_args) File "/home/ubuntu/Documents/sglang/python/sglang/srt/entrypoints/http_server.py", line 1692, in launch_server _launch_subprocesses( File "/home/ubuntu/Documents/sglang/python/sglang/srt/entrypoints/engine.py", line 966, in _launch_subprocesses data = scheduler_pipe_readers[i].recv() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() ^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/connection.py", line 430, in _recv_bytes buf = self._recv(4) ^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/kt-kernel/lib/python3.11/multiprocessing/connection.py", line 399, in _recv raise EOFError EOFError ``` ### Others _No response_
bsakaguchi commented 2026-10-29 14:48:40 +00:00
Author
Owner
Copy link

@ovowei commented on GitHub (Jan 22, 2026):

please try the latest version

<!-- gh-comment-id:3783937137 --> @ovowei commented on GitHub (Jan 22, 2026): please try the latest version
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#1109
No description provided.