[GH-ISSUE #1750] terminate called after throwing an instance of 'std::logic_error' #1103

New issue

Closed

opened 2026-10-29 14:48:34 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:48:34 +00:00

Owner

Originally created by @sunnsi on GitHub (Dec 25, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1750

Originally assigned to: @KMSorSMS on GitHub.

Reminder

I have read the above rules and searched the existing issues.

System Info

KTransformers 环境诊断

┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Check ┃ Status ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Python 版本 │ 正常 │ 3.11.11 │
│ CUDA 可用性 │ 正常 │ 12.6 │
│ GPU 检测 │ 正常 │ 发现 1 个 GPU: NVIDIA RTX A6000 │
│ CPU │ 正常 │ INTEL(R) XEON(R) GOLD 6530 (64 核心 / 64 线程) │
│ CPU 指令集 │ 正常 │ AMX-INT8, AMX-BF16, AMX-TILE, AVX512BF16, AVX512F, AVX512BW, AVX512VL, AVX2 (+14 more) │
│ NUMA 拓扑 │ 正常 │ 4 个节点 │
│ 系统内存 │ 正常 │ 592.5GB 可用 / 628.6GB 总计 │
│ 磁盘空间 │ 正常 │ /data/llm-models 有 2563.4GB 可用空间 │
│ SGLang Source │ 正常 │ Source (GitHub: kvcache-ai/sglang, branch: main) │
│ SGLang kt-kernel │ 正常 │ SGLang kt-kernel 支持已验证 │
│ Environment Managers │ 正常 │ conda 24.9.2, venv builtin, docker 28.3.0 │
└──────────────────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────┘

✓ 所有检查通过！您的环境已就绪。

Reproduction

python -m sglang.launch_server \
  --host 0.0.0.0 \
  --port 10002 \
  --model /data/llm-models/KVCache-ai/Kimi-K2-Thinking-CPU-weight \
  --trust-remote-code \
  --mem-fraction-static 0.92 \
  --chunked-prefill-size 4096 \
  --served-model-name kimi-k2 \
  --enable-mixed-chunk \
  --kt-method AMXINT4 \
  --kt-weight-path /data/llm-models/KVCache-ai/Kimi-K2-Thinking-CPU-weight/model-00001.safetensors \
  --kt-cpuinfer 64 \
  --kt-threadpool-count 4 \
  --kt-num-gpu-experts 12 \
  --kt-max-deferred-experts-per-token 2

Others

Error:

Loading safetensors checkpoint shards: 100% Completed | 60/60 [31:13<00:00, 31.11s/it]
Loading safetensors checkpoint shards: 100% Completed | 60/60 [31:13<00:00, 31.22s/it]

TP MOE layer 1, pool: 0x35f06890, expert num: 384, num_experts_per_tok: 8
TP Load from file
terminate called after throwing an instance of 'std::logic_error'
terminate called after throwing an instance of 'std::logic_error'
what(): basic_string::_M_construct null not valid
terminate called after throwing an instance of 'std::logic_error'
Fatal Python error: terminate called recursively
Aborted

Thread 0x what(): 00007fa649fff6c0basic_string::_M_construct null not valid (most recent call first):

File what(): basic_string::_M_construct null not valid
[2025-12-25 15:18:18] Rank 0 scheduler is dead. Please check if there are relevant logs.
[2025-12-25 15:18:18] Exit code: -6
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/mgi527a/Softwares/sglang/python/sglang/launch_server.py", line 32, in
run_server(server_args)
File "/home/mgi527a/Softwares/sglang/python/sglang/launch_server.py", line 25, in run_server
launch_server(server_args)
File "/home/mgi527a/Softwares/sglang/python/sglang/srt/entrypoints/http_server.py", line 1692, in launch_server
_launch_subprocesses(
File "/home/mgi527a/Softwares/sglang/python/sglang/srt/entrypoints/engine.py", line 966, in _launch_subprocesses
data = scheduler_pipe_readers[i].recv()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mgi527a/anaconda3/envs/ktransformers/lib/python3.11/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
^^^^^^^^^^^^^^^^^^
File "/home/mgi527a/anaconda3/envs/ktransformers/lib/python3.11/multiprocessing/connection.py", line 430, in _recv_bytes
buf = self._recv(4)
^^^^^^^^^^^^^
File "/home/mgi527a/anaconda3/envs/ktransformers/lib/python3.11/multiprocessing/connection.py", line 399, in _recv
raise EOFError
EOFError

Originally created by @sunnsi on GitHub (Dec 25, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1750 Originally assigned to: @KMSorSMS on GitHub. ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info KTransformers 环境诊断 ┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Check ┃ Status ┃ Value ┃ ┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ Python 版本 │ 正常 │ 3.11.11 │ │ CUDA 可用性 │ 正常 │ 12.6 │ │ GPU 检测 │ 正常 │ 发现 1 个 GPU: NVIDIA RTX A6000 │ │ CPU │ 正常 │ INTEL(R) XEON(R) GOLD 6530 (64 核心 / 64 线程) │ │ CPU 指令集 │ 正常 │ AMX-INT8, AMX-BF16, AMX-TILE, AVX512BF16, AVX512F, AVX512BW, AVX512VL, AVX2 (+14 more) │ │ NUMA 拓扑 │ 正常 │ 4 个节点 │ │ 系统内存 │ 正常 │ 592.5GB 可用 / 628.6GB 总计 │ │ 磁盘空间 │ 正常 │ /data/llm-models 有 2563.4GB 可用空间 │ │ SGLang Source │ 正常 │ Source (GitHub: kvcache-ai/sglang, branch: main) │ │ SGLang kt-kernel │ 正常 │ SGLang kt-kernel 支持已验证 │ │ Environment Managers │ 正常 │ conda 24.9.2, venv builtin, docker 28.3.0 │ └──────────────────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────┘ ✓ 所有检查通过！您的环境已就绪。 ### Reproduction ```text python -m sglang.launch_server \ --host 0.0.0.0 \ --port 10002 \ --model /data/llm-models/KVCache-ai/Kimi-K2-Thinking-CPU-weight \ --trust-remote-code \ --mem-fraction-static 0.92 \ --chunked-prefill-size 4096 \ --served-model-name kimi-k2 \ --enable-mixed-chunk \ --kt-method AMXINT4 \ --kt-weight-path /data/llm-models/KVCache-ai/Kimi-K2-Thinking-CPU-weight/model-00001.safetensors \ --kt-cpuinfer 64 \ --kt-threadpool-count 4 \ --kt-num-gpu-experts 12 \ --kt-max-deferred-experts-per-token 2 ``` ### Others Error: Loading safetensors checkpoint shards: 100% Completed | 60/60 [31:13<00:00, 31.11s/it] Loading safetensors checkpoint shards: 100% Completed | 60/60 [31:13<00:00, 31.22s/it] TP MOE layer 1, pool: 0x35f06890, expert num: 384, num_experts_per_tok: 8 TP Load from file terminate called after throwing an instance of 'std::logic_error' terminate called after throwing an instance of 'std::logic_error' what(): basic_string::_M_construct null not valid terminate called after throwing an instance of 'std::logic_error' Fatal Python error: terminate called recursively Aborted Thread 0x what(): 00007fa649fff6c0basic_string::_M_construct null not valid (most recent call first): File what(): basic_string::_M_construct null not valid [2025-12-25 15:18:18] Rank 0 scheduler is dead. Please check if there are relevant logs. [2025-12-25 15:18:18] Exit code: -6 Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/home/mgi527a/Softwares/sglang/python/sglang/launch_server.py", line 32, in <module> run_server(server_args) File "/home/mgi527a/Softwares/sglang/python/sglang/launch_server.py", line 25, in run_server launch_server(server_args) File "/home/mgi527a/Softwares/sglang/python/sglang/srt/entrypoints/http_server.py", line 1692, in launch_server _launch_subprocesses( File "/home/mgi527a/Softwares/sglang/python/sglang/srt/entrypoints/engine.py", line 966, in _launch_subprocesses data = scheduler_pipe_readers[i].recv() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mgi527a/anaconda3/envs/ktransformers/lib/python3.11/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() ^^^^^^^^^^^^^^^^^^ File "/home/mgi527a/anaconda3/envs/ktransformers/lib/python3.11/multiprocessing/connection.py", line 430, in _recv_bytes buf = self._recv(4) ^^^^^^^^^^^^^ File "/home/mgi527a/anaconda3/envs/ktransformers/lib/python3.11/multiprocessing/connection.py", line 399, in _recv raise EOFError EOFError

bsakaguchi

2026-10-29 14:48:34 +00:00