[GH-ISSUE #1713] SIGILL on Cascade Lake #1093

New issue

Closed

opened 2026-10-29 14:48:25 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:48:25 +00:00

Owner

Originally created by @zymurgy on GitHub (Dec 13, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1713

Originally assigned to: @KMSorSMS on GitHub.

Reminder

I have read the above rules and searched the existing issues.

System Info

Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             46 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      64
  On-line CPU(s) list:       0-63
Vendor ID:                   GenuineIntel
  BIOS Vendor ID:            Intel(R) Corporation
  Model name:                Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
    BIOS Model name:         Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz  CPU @ 2.1GHz
    BIOS CPU family:         179
    CPU family:              6
    Model:                   85
    Thread(s) per core:      2
    Core(s) per socket:      16
    Socket(s):               2
    Stepping:                7
    CPU(s) scaling MHz:      97%
    CPU max MHz:             3200.0000
    CPU min MHz:             800.0000
    BogoMIPS:                4200.00
    Flags:                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscal
                             l nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulq
                             dq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline
                             _timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 intel_ppin ssbd mba ibrs ibpb stibp i
                             brs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512
                             f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occu
                             p_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi pku ospke avx512_vnni md_clear flush_l1d arch_capabilities
Virtualization features:
  Virtualization:            VT-x
Caches (sum of all):
  L1d:                       1 MiB (32 instances)
  L1i:                       1 MiB (32 instances)
  L2:                        32 MiB (32 instances)
  L3:                        44 MiB (2 instances)
NUMA:
  NUMA node(s):              2
  NUMA node0 CPU(s):         0-15,32-47
  NUMA node1 CPU(s):         16-31,48-63
Vulnerabilities:
  Gather data sampling:      Vulnerable
  Ghostwrite:                Not affected
  Indirect target selection: Mitigation; Aligned branch/return thunks
  Itlb multihit:             KVM: Mitigation: Split huge pages
  L1tf:                      Not affected
  Mds:                       Not affected
  Meltdown:                  Not affected
  Mmio stale data:           Mitigation; Clear CPU buffers; SMT vulnerable
  Reg file data sampling:    Not affected
  Retbleed:                  Mitigation; Enhanced IBRS
  Spec rstack overflow:      Not affected
  Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:                Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI SW loop, KVM SW loop
  Srbds:                     Not affected
  Tsx async abort:           Mitigation; TSX disabled

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.82.07              Driver Version: 580.82.07      CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090        On  |   00000000:D8:00.0 Off |                  N/A |
|  0%   30C    P8             15W /  250W |       0MiB /  24576MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

System is a dual Xeon Silver 4216 and a single RTX 3090. When I build kt-kernel for Cascade Lake:

export CPUINFER_CPU_INSTRUCT=AVX512
export CPUINFER_ENABLE_AMX=OFF

./install.sh deps && ./install.sh build --manual

I get a SIGILL when loading the model. I tracked this down to kt-kernel trying to execute a vcvtne2ps2bf16 instruction which is not supported on Cascade Lake. It looks like -mavx512bf16 is being passed during the build phase? I changed CPUINFER_CPU_INSTRUCT=AVX2 and the model loaded with no issues, but would like to test the performance benefits of AVX512 on Cascade Lake.

warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libcap.so.2
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `sglang::scheduler                                                             '.
Program terminated with signal SIGILL, Illegal instruction.
#0  0x00007c7f24c4eb2c in pthread_kill () from /lib/x86_64-linux-gnu/libc.so.6
[Current thread is 1 (Thread 0x7c6a23ffe6c0 (LWP 569))]
(gdb) bt
#0  0x00007c7f24c4eb2c in pthread_kill () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007c7f24bf527e in raise () from /lib/x86_64-linux-gnu/libc.so.6
#2  <signal handler called>
#3  0x00007c7aa49f1beb in ggml_fp32_to_bf16_row ()
   from /usr/local/lib/python3.12/dist-packages/kt_kernel_ext.cpython-312-x86_64-linux-gnu.so
#4  0x00007c7aa4851cfe in ?? () from /usr/local/lib/python3.12/dist-packages/kt_kernel_ext.cpython-312-x86_64-linux-gnu.so
#5  0x00007c7aa48523db in ?? () from /usr/local/lib/python3.12/dist-packages/kt_kernel_ext.cpython-312-x86_64-linux-gnu.so
#6  0x00007c7f22085db4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007c7f24c4caa4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#8  0x00007c7f24cd9a64 in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) frame 3
#3  0x00007c7aa49f1beb in ggml_fp32_to_bf16_row ()
   from /usr/local/lib/python3.12/dist-packages/kt_kernel_ext.cpython-312-x86_64-linux-gnu.so
(gdb) info registers
rax            0x0                 0
rbx            0xa                 10
rcx            0x7e0               2016
rdx            0x0                 0
rsi            0x7e0               2016
rdi            0x800               2048
rbp            0x7c6a23ffddf0      0x7c6a23ffddf0
rsp            0x7c6a23ffddd8      0x7c6a23ffddd8
r8             0x48896db0          1216966064
r9             0x4863dbb0          1214503856
r10            0x7c6456c49040      136770394296384
r11            0x7c7a8644ba00      136865680505344
r12            0x7c6a23ffde14      136795312348692
r13            0x7c6a40003380      136795782132608
r14            0x7c6a40029fc0      136795782291392
r15            0x7c6a400032f0      136795782132464
rip            0x7c7aa49f1beb      0x7c7aa49f1beb <ggml_fp32_to_bf16_row+59>
eflags         0x246               [ PF ZF IF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0
k0             0x80000             524288
k1             0x1f                31
k2             0x3f                63
k3             0x0                 0
k4             0x0                 0
k5             0x0                 0
k6             0x0                 0
k7             0x0                 0
fs_base        0x7c6a23ffe6c0      136795312350912
gs_base        0x0                 0
(gdb) p/x $pc
$1 = 0x7c7aa49f1beb
(gdb) x/16i $pc
=> 0x7c7aa49f1beb <ggml_fp32_to_bf16_row+59>:	vcvtne2ps2bf16 (%r10,%rax,4),%zmm7,%zmm0
   0x7c7aa49f1bf2 <ggml_fp32_to_bf16_row+66>:	vmovdqu64 %zmm0,(%r11,%rax,2)
   0x7c7aa49f1bf9 <ggml_fp32_to_bf16_row+73>:	add    $0x20,%rax
   0x7c7aa49f1bfd <ggml_fp32_to_bf16_row+77>:	cmp    %rdx,%rcx
   0x7c7aa49f1c00 <ggml_fp32_to_bf16_row+80>:	jne    0x7c7aa49f1be0 <ggml_fp32_to_bf16_row+48>
   0x7c7aa49f1c02 <ggml_fp32_to_bf16_row+82>:	shr    $0x5,%rsi
   0x7c7aa49f1c06 <ggml_fp32_to_bf16_row+86>:	lea    0x1(%rsi),%eax
   0x7c7aa49f1c09 <ggml_fp32_to_bf16_row+89>:	shl    $0x5,%eax
   0x7c7aa49f1c0c <ggml_fp32_to_bf16_row+92>:	movslq %eax,%rcx
   0x7c7aa49f1c0f <ggml_fp32_to_bf16_row+95>:	cmp    %rcx,%rdi
   0x7c7aa49f1c12 <ggml_fp32_to_bf16_row+98>:	jle    0x7c7aa49f1f5b <ggml_fp32_to_bf16_row+939>
   0x7c7aa49f1c18 <ggml_fp32_to_bf16_row+104>:	mov    %rdi,%r9
   0x7c7aa49f1c1b <ggml_fp32_to_bf16_row+107>:	sub    %rcx,%r9
   0x7c7aa49f1c1e <ggml_fp32_to_bf16_row+110>:	lea    -0x1(%r9),%rdx
   0x7c7aa49f1c22 <ggml_fp32_to_bf16_row+114>:	mov    %r9,%r8
   0x7c7aa49f1c25 <ggml_fp32_to_bf16_row+117>:	cmp    $0x1e,%rdx
(gdb) x/64bx $pc
0x7c7aa49f1beb <ggml_fp32_to_bf16_row+59>:	0x62	0xd2	0x47	0x48	0x72	0x04	0x82	0x62
0x7c7aa49f1bf3 <ggml_fp32_to_bf16_row+67>:	0xd1	0xfe	0x48	0x7f	0x04	0x43	0x48	0x83
0x7c7aa49f1bfb <ggml_fp32_to_bf16_row+75>:	0xc0	0x20	0x48	0x39	0xd1	0x75	0xde	0x48
0x7c7aa49f1c03 <ggml_fp32_to_bf16_row+83>:	0xc1	0xee	0x05	0x8d	0x46	0x01	0xc1	0xe0
0x7c7aa49f1c0b <ggml_fp32_to_bf16_row+91>:	0x05	0x48	0x63	0xc8	0x48	0x39	0xcf	0x0f
0x7c7aa49f1c13 <ggml_fp32_to_bf16_row+99>:	0x8e	0x43	0x03	0x00	0x00	0x49	0x89	0xf9
0x7c7aa49f1c1b <ggml_fp32_to_bf16_row+107>:	0x49	0x29	0xc9	0x49	0x8d	0x51	0xff	0x4d
0x7c7aa49f1c23 <ggml_fp32_to_bf16_row+115>:	0x89	0xc8	0x48	0x83	0xfa	0x1e	0x0f	0x86
(gdb) disassemble /r $pc-32, $pc+64
Dump of assembler code from 0x7c7aa49f1bcb to 0x7c7aa49f1c2b:
   0x00007c7aa49f1bcb <ggml_fp32_to_bf16_row+27>:	8e 98 03 00 00 48  	mov    0x48000003(%rax),%ds
   0x00007c7aa49f1bd1 <ggml_fp32_to_bf16_row+33>:	8d 72 e0           	lea    -0x20(%rdx),%esi
   0x00007c7aa49f1bd4 <ggml_fp32_to_bf16_row+36>:	31 c0              	xor    %eax,%eax
   0x00007c7aa49f1bd6 <ggml_fp32_to_bf16_row+38>:	48 89 f1           	mov    %rsi,%rcx
   0x00007c7aa49f1bd9 <ggml_fp32_to_bf16_row+41>:	48 83 e1 e0        	and    $0xffffffffffffffe0,%rcx
   0x00007c7aa49f1bdd <ggml_fp32_to_bf16_row+45>:	0f 1f 00           	nopl   (%rax)
   0x00007c7aa49f1be0 <ggml_fp32_to_bf16_row+48>:	62 d1 7c 48 10 7c 82 01	vmovups 0x40(%r10,%rax,4),%zmm7
   0x00007c7aa49f1be8 <ggml_fp32_to_bf16_row+56>:	48 89 c2           	mov    %rax,%rdx
=> 0x00007c7aa49f1beb <ggml_fp32_to_bf16_row+59>:	62 d2 47 48 72 04 82	vcvtne2ps2bf16 (%r10,%rax,4),%zmm7,%zmm0
   0x00007c7aa49f1bf2 <ggml_fp32_to_bf16_row+66>:	62 d1 fe 48 7f 04 43	vmovdqu64 %zmm0,(%r11,%rax,2)
   0x00007c7aa49f1bf9 <ggml_fp32_to_bf16_row+73>:	48 83 c0 20        	add    $0x20,%rax
   0x00007c7aa49f1bfd <ggml_fp32_to_bf16_row+77>:	48 39 d1           	cmp    %rdx,%rcx
   0x00007c7aa49f1c00 <ggml_fp32_to_bf16_row+80>:	75 de              	jne    0x7c7aa49f1be0 <ggml_fp32_to_bf16_row+48>
   0x00007c7aa49f1c02 <ggml_fp32_to_bf16_row+82>:	48 c1 ee 05        	shr    $0x5,%rsi
   0x00007c7aa49f1c06 <ggml_fp32_to_bf16_row+86>:	8d 46 01           	lea    0x1(%rsi),%eax
   0x00007c7aa49f1c09 <ggml_fp32_to_bf16_row+89>:	c1 e0 05           	shl    $0x5,%eax
   0x00007c7aa49f1c0c <ggml_fp32_to_bf16_row+92>:	48 63 c8           	movslq %eax,%rcx
   0x00007c7aa49f1c0f <ggml_fp32_to_bf16_row+95>:	48 39 cf           	cmp    %rcx,%rdi
   0x00007c7aa49f1c12 <ggml_fp32_to_bf16_row+98>:	0f 8e 43 03 00 00  	jle    0x7c7aa49f1f5b <ggml_fp32_to_bf16_row+939>
   0x00007c7aa49f1c18 <ggml_fp32_to_bf16_row+104>:	49 89 f9           	mov    %rdi,%r9
   0x00007c7aa49f1c1b <ggml_fp32_to_bf16_row+107>:	49 29 c9           	sub    %rcx,%r9
   0x00007c7aa49f1c1e <ggml_fp32_to_bf16_row+110>:	49 8d 51 ff        	lea    -0x1(%r9),%rdx
   0x00007c7aa49f1c22 <ggml_fp32_to_bf16_row+114>:	4d 89 c8           	mov    %r9,%r8
   0x00007c7aa49f1c25 <ggml_fp32_to_bf16_row+117>:	48 83 fa 1e        	cmp    $0x1e,%rdx
   0x00007c7aa49f1c29 <ggml_fp32_to_bf16_row+121>:	0f 86 4c 03 00 00  	jbe    0x7c7aa49f1f7b <ggml_fp32_to_bf16_row+971>
End of assembler dump.

Reproduction

FROM lmsysorg/sglang:latest

ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1

RUN apt-get update && apt-get install -y --no-install-recommends \
    git \
    build-essential \
    cmake \
    ninja-build \
    pkg-config \
    hwloc \
    libhwloc-dev \
    numactl \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /opt

RUN git clone --recursive https://github.com/kvcache-ai/ktransformers.git

WORKDIR /opt/ktransformers/kt-kernel

ENV CPUINFER_CPU_INSTRUCT=AVX512 \
    CPUINFER_ENABLE_AMX=OFF

RUN ./install.sh deps && ./install.sh build --manual

EXPOSE 8000

WORKDIR /workspace

CMD ["python", "-m", "sglang.launch_server", \
  "--host", "0.0.0.0", \
  "--port", "8000", \
  "--model", "/models/Qwen/Qwen3-30B-A3B", \
  "--trust-remote-code", \
  "--mem-fraction-static", "0.92", \
  "--chunked-prefill-size", "4096", \
  "--served-model-name", "Qwen3-30B-A3B", \
  "--enable-mixed-chunk", \
  "--kt-method", "LLAMAFILE", \
  "--kt-weight-path", "/workspace/models", \
  "--kt-cpuinfer", "32", \
  "--kt-threadpool-count", "2", \
  "--kt-num-gpu-experts", "32", \
  "--kt-max-deferred-experts-per-token", "2" \
]

Others

No response

Originally created by @zymurgy on GitHub (Dec 13, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1713 Originally assigned to: @KMSorSMS on GitHub. ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info ``` Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 46 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 64 On-line CPU(s) list: 0-63 Vendor ID: GenuineIntel BIOS Vendor ID: Intel(R) Corporation Model name: Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz BIOS Model name: Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz CPU @ 2.1GHz BIOS CPU family: 179 CPU family: 6 Model: 85 Thread(s) per core: 2 Core(s) per socket: 16 Socket(s): 2 Stepping: 7 CPU(s) scaling MHz: 97% CPU max MHz: 3200.0000 CPU min MHz: 800.0000 BogoMIPS: 4200.00 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscal l nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulq dq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline _timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 intel_ppin ssbd mba ibrs ibpb stibp i brs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512 f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occu p_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi pku ospke avx512_vnni md_clear flush_l1d arch_capabilities Virtualization features: Virtualization: VT-x Caches (sum of all): L1d: 1 MiB (32 instances) L1i: 1 MiB (32 instances) L2: 32 MiB (32 instances) L3: 44 MiB (2 instances) NUMA: NUMA node(s): 2 NUMA node0 CPU(s): 0-15,32-47 NUMA node1 CPU(s): 16-31,48-63 Vulnerabilities: Gather data sampling: Vulnerable Ghostwrite: Not affected Indirect target selection: Mitigation; Aligned branch/return thunks Itlb multihit: KVM: Mitigation: Split huge pages L1tf: Not affected Mds: Not affected Meltdown: Not affected Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable Reg file data sampling: Not affected Retbleed: Mitigation; Enhanced IBRS Spec rstack overflow: Not affected Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI SW loop, KVM SW loop Srbds: Not affected Tsx async abort: Mitigation; TSX disabled ``` ``` +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 580.82.07 Driver Version: 580.82.07 CUDA Version: 13.0 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 3090 On | 00000000:D8:00.0 Off | N/A | | 0% 30C P8 15W / 250W | 0MiB / 24576MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+ ``` System is a dual Xeon Silver 4216 and a single RTX 3090. When I build kt-kernel for Cascade Lake: export CPUINFER_CPU_INSTRUCT=AVX512 export CPUINFER_ENABLE_AMX=OFF ./install.sh deps && ./install.sh build --manual I get a SIGILL when loading the model. I tracked this down to kt-kernel trying to execute a vcvtne2ps2bf16 instruction which is not supported on Cascade Lake. It looks like -mavx512bf16 is being passed during the build phase? I changed CPUINFER_CPU_INSTRUCT=AVX2 and the model loaded with no issues, but would like to test the performance benefits of AVX512 on Cascade Lake. ``` warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libcap.so.2 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `sglang::scheduler '. Program terminated with signal SIGILL, Illegal instruction. #0 0x00007c7f24c4eb2c in pthread_kill () from /lib/x86_64-linux-gnu/libc.so.6 [Current thread is 1 (Thread 0x7c6a23ffe6c0 (LWP 569))] (gdb) bt #0 0x00007c7f24c4eb2c in pthread_kill () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007c7f24bf527e in raise () from /lib/x86_64-linux-gnu/libc.so.6 #2 <signal handler called> #3 0x00007c7aa49f1beb in ggml_fp32_to_bf16_row () from /usr/local/lib/python3.12/dist-packages/kt_kernel_ext.cpython-312-x86_64-linux-gnu.so #4 0x00007c7aa4851cfe in ?? () from /usr/local/lib/python3.12/dist-packages/kt_kernel_ext.cpython-312-x86_64-linux-gnu.so #5 0x00007c7aa48523db in ?? () from /usr/local/lib/python3.12/dist-packages/kt_kernel_ext.cpython-312-x86_64-linux-gnu.so #6 0x00007c7f22085db4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6 #7 0x00007c7f24c4caa4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #8 0x00007c7f24cd9a64 in clone () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) frame 3 #3 0x00007c7aa49f1beb in ggml_fp32_to_bf16_row () from /usr/local/lib/python3.12/dist-packages/kt_kernel_ext.cpython-312-x86_64-linux-gnu.so (gdb) info registers rax 0x0 0 rbx 0xa 10 rcx 0x7e0 2016 rdx 0x0 0 rsi 0x7e0 2016 rdi 0x800 2048 rbp 0x7c6a23ffddf0 0x7c6a23ffddf0 rsp 0x7c6a23ffddd8 0x7c6a23ffddd8 r8 0x48896db0 1216966064 r9 0x4863dbb0 1214503856 r10 0x7c6456c49040 136770394296384 r11 0x7c7a8644ba00 136865680505344 r12 0x7c6a23ffde14 136795312348692 r13 0x7c6a40003380 136795782132608 r14 0x7c6a40029fc0 136795782291392 r15 0x7c6a400032f0 136795782132464 rip 0x7c7aa49f1beb 0x7c7aa49f1beb <ggml_fp32_to_bf16_row+59> eflags 0x246 [ PF ZF IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 k0 0x80000 524288 k1 0x1f 31 k2 0x3f 63 k3 0x0 0 k4 0x0 0 k5 0x0 0 k6 0x0 0 k7 0x0 0 fs_base 0x7c6a23ffe6c0 136795312350912 gs_base 0x0 0 (gdb) p/x $pc $1 = 0x7c7aa49f1beb (gdb) x/16i $pc => 0x7c7aa49f1beb <ggml_fp32_to_bf16_row+59>: vcvtne2ps2bf16 (%r10,%rax,4),%zmm7,%zmm0 0x7c7aa49f1bf2 <ggml_fp32_to_bf16_row+66>: vmovdqu64 %zmm0,(%r11,%rax,2) 0x7c7aa49f1bf9 <ggml_fp32_to_bf16_row+73>: add $0x20,%rax 0x7c7aa49f1bfd <ggml_fp32_to_bf16_row+77>: cmp %rdx,%rcx 0x7c7aa49f1c00 <ggml_fp32_to_bf16_row+80>: jne 0x7c7aa49f1be0 <ggml_fp32_to_bf16_row+48> 0x7c7aa49f1c02 <ggml_fp32_to_bf16_row+82>: shr $0x5,%rsi 0x7c7aa49f1c06 <ggml_fp32_to_bf16_row+86>: lea 0x1(%rsi),%eax 0x7c7aa49f1c09 <ggml_fp32_to_bf16_row+89>: shl $0x5,%eax 0x7c7aa49f1c0c <ggml_fp32_to_bf16_row+92>: movslq %eax,%rcx 0x7c7aa49f1c0f <ggml_fp32_to_bf16_row+95>: cmp %rcx,%rdi 0x7c7aa49f1c12 <ggml_fp32_to_bf16_row+98>: jle 0x7c7aa49f1f5b <ggml_fp32_to_bf16_row+939> 0x7c7aa49f1c18 <ggml_fp32_to_bf16_row+104>: mov %rdi,%r9 0x7c7aa49f1c1b <ggml_fp32_to_bf16_row+107>: sub %rcx,%r9 0x7c7aa49f1c1e <ggml_fp32_to_bf16_row+110>: lea -0x1(%r9),%rdx 0x7c7aa49f1c22 <ggml_fp32_to_bf16_row+114>: mov %r9,%r8 0x7c7aa49f1c25 <ggml_fp32_to_bf16_row+117>: cmp $0x1e,%rdx (gdb) x/64bx $pc 0x7c7aa49f1beb <ggml_fp32_to_bf16_row+59>: 0x62 0xd2 0x47 0x48 0x72 0x04 0x82 0x62 0x7c7aa49f1bf3 <ggml_fp32_to_bf16_row+67>: 0xd1 0xfe 0x48 0x7f 0x04 0x43 0x48 0x83 0x7c7aa49f1bfb <ggml_fp32_to_bf16_row+75>: 0xc0 0x20 0x48 0x39 0xd1 0x75 0xde 0x48 0x7c7aa49f1c03 <ggml_fp32_to_bf16_row+83>: 0xc1 0xee 0x05 0x8d 0x46 0x01 0xc1 0xe0 0x7c7aa49f1c0b <ggml_fp32_to_bf16_row+91>: 0x05 0x48 0x63 0xc8 0x48 0x39 0xcf 0x0f 0x7c7aa49f1c13 <ggml_fp32_to_bf16_row+99>: 0x8e 0x43 0x03 0x00 0x00 0x49 0x89 0xf9 0x7c7aa49f1c1b <ggml_fp32_to_bf16_row+107>: 0x49 0x29 0xc9 0x49 0x8d 0x51 0xff 0x4d 0x7c7aa49f1c23 <ggml_fp32_to_bf16_row+115>: 0x89 0xc8 0x48 0x83 0xfa 0x1e 0x0f 0x86 (gdb) disassemble /r $pc-32, $pc+64 Dump of assembler code from 0x7c7aa49f1bcb to 0x7c7aa49f1c2b: 0x00007c7aa49f1bcb <ggml_fp32_to_bf16_row+27>: 8e 98 03 00 00 48 mov 0x48000003(%rax),%ds 0x00007c7aa49f1bd1 <ggml_fp32_to_bf16_row+33>: 8d 72 e0 lea -0x20(%rdx),%esi 0x00007c7aa49f1bd4 <ggml_fp32_to_bf16_row+36>: 31 c0 xor %eax,%eax 0x00007c7aa49f1bd6 <ggml_fp32_to_bf16_row+38>: 48 89 f1 mov %rsi,%rcx 0x00007c7aa49f1bd9 <ggml_fp32_to_bf16_row+41>: 48 83 e1 e0 and $0xffffffffffffffe0,%rcx 0x00007c7aa49f1bdd <ggml_fp32_to_bf16_row+45>: 0f 1f 00 nopl (%rax) 0x00007c7aa49f1be0 <ggml_fp32_to_bf16_row+48>: 62 d1 7c 48 10 7c 82 01 vmovups 0x40(%r10,%rax,4),%zmm7 0x00007c7aa49f1be8 <ggml_fp32_to_bf16_row+56>: 48 89 c2 mov %rax,%rdx => 0x00007c7aa49f1beb <ggml_fp32_to_bf16_row+59>: 62 d2 47 48 72 04 82 vcvtne2ps2bf16 (%r10,%rax,4),%zmm7,%zmm0 0x00007c7aa49f1bf2 <ggml_fp32_to_bf16_row+66>: 62 d1 fe 48 7f 04 43 vmovdqu64 %zmm0,(%r11,%rax,2) 0x00007c7aa49f1bf9 <ggml_fp32_to_bf16_row+73>: 48 83 c0 20 add $0x20,%rax 0x00007c7aa49f1bfd <ggml_fp32_to_bf16_row+77>: 48 39 d1 cmp %rdx,%rcx 0x00007c7aa49f1c00 <ggml_fp32_to_bf16_row+80>: 75 de jne 0x7c7aa49f1be0 <ggml_fp32_to_bf16_row+48> 0x00007c7aa49f1c02 <ggml_fp32_to_bf16_row+82>: 48 c1 ee 05 shr $0x5,%rsi 0x00007c7aa49f1c06 <ggml_fp32_to_bf16_row+86>: 8d 46 01 lea 0x1(%rsi),%eax 0x00007c7aa49f1c09 <ggml_fp32_to_bf16_row+89>: c1 e0 05 shl $0x5,%eax 0x00007c7aa49f1c0c <ggml_fp32_to_bf16_row+92>: 48 63 c8 movslq %eax,%rcx 0x00007c7aa49f1c0f <ggml_fp32_to_bf16_row+95>: 48 39 cf cmp %rcx,%rdi 0x00007c7aa49f1c12 <ggml_fp32_to_bf16_row+98>: 0f 8e 43 03 00 00 jle 0x7c7aa49f1f5b <ggml_fp32_to_bf16_row+939> 0x00007c7aa49f1c18 <ggml_fp32_to_bf16_row+104>: 49 89 f9 mov %rdi,%r9 0x00007c7aa49f1c1b <ggml_fp32_to_bf16_row+107>: 49 29 c9 sub %rcx,%r9 0x00007c7aa49f1c1e <ggml_fp32_to_bf16_row+110>: 49 8d 51 ff lea -0x1(%r9),%rdx 0x00007c7aa49f1c22 <ggml_fp32_to_bf16_row+114>: 4d 89 c8 mov %r9,%r8 0x00007c7aa49f1c25 <ggml_fp32_to_bf16_row+117>: 48 83 fa 1e cmp $0x1e,%rdx 0x00007c7aa49f1c29 <ggml_fp32_to_bf16_row+121>: 0f 86 4c 03 00 00 jbe 0x7c7aa49f1f7b <ggml_fp32_to_bf16_row+971> End of assembler dump. ``` ### Reproduction ``` FROM lmsysorg/sglang:latest ENV DEBIAN_FRONTEND=noninteractive \ PYTHONUNBUFFERED=1 \ PIP_NO_CACHE_DIR=1 RUN apt-get update && apt-get install -y --no-install-recommends \ git \ build-essential \ cmake \ ninja-build \ pkg-config \ hwloc \ libhwloc-dev \ numactl \ && rm -rf /var/lib/apt/lists/* WORKDIR /opt RUN git clone --recursive https://github.com/kvcache-ai/ktransformers.git WORKDIR /opt/ktransformers/kt-kernel ENV CPUINFER_CPU_INSTRUCT=AVX512 \ CPUINFER_ENABLE_AMX=OFF RUN ./install.sh deps && ./install.sh build --manual EXPOSE 8000 WORKDIR /workspace CMD ["python", "-m", "sglang.launch_server", \ "--host", "0.0.0.0", \ "--port", "8000", \ "--model", "/models/Qwen/Qwen3-30B-A3B", \ "--trust-remote-code", \ "--mem-fraction-static", "0.92", \ "--chunked-prefill-size", "4096", \ "--served-model-name", "Qwen3-30B-A3B", \ "--enable-mixed-chunk", \ "--kt-method", "LLAMAFILE", \ "--kt-weight-path", "/workspace/models", \ "--kt-cpuinfer", "32", \ "--kt-threadpool-count", "2", \ "--kt-num-gpu-experts", "32", \ "--kt-max-deferred-experts-per-token", "2" \ ] ``` ### Others _No response_

bsakaguchi

2026-10-29 14:48:25 +00:00