kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1782] Conver DeepSeek V3.2 to CPU weight failed #1122

New issue
Closed
opened 2026-10-29 14:48:46 +00:00 by bsakaguchi · 7 comments
bsakaguchi commented 2026-10-29 14:48:46 +00:00
Owner
Copy link

Originally created by @mrgaolei on GitHub (Jan 4, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1782

Originally assigned to: @ouqingliang on GitHub.

Reminder

  • I have read the above rules and searched the existing issues.

System Info

kt-kernel 0.5.0.post1
CPU: 2 x 6416h
RAM: 1T DDR5
GPU: 4090 48G

Ubuntu 24.04
CUDA 13.0

Reproduction

convert script

n scripts/convert_cpu_weights.py -i /opt/ai-models/DeepSeek-V3.2 --input-type fp8 --output /opt/ai-models/DeepSeek-V3.2-INT4 --quant-method int4 --cpuinfer-threads 10 --threadpool-count 1 --no-merge-safetensor```

but throw this error:
```bash
  Layer 59: 256 experts (0-255)
  Layer 60: 256 experts (0-255)
  Layer 61: 256 experts (0-255)
Processing layer 3 (1/59)...
Converting layer 3 with 256 experts via online quantization...
  Loaded weights shapes:
    gate_proj: torch.Size([256, 2048, 7168])
    up_proj: torch.Size([256, 2048, 7168])
    down_proj: torch.Size([256, 7168, 2048])
CPUInfer[0x43564070]: Hello
WorkerPool[0x43625890] 1 subpools, [numa:threads][0:10]
===========In NumaPool============
In Numa Worker Pool at NUMA 0, 10 threads
TP MOE layer 3, pool: 0x43625890, expert num: 256, num_experts_per_tok: 8
From BF16
  online quant from bf16
段错误 (核心已转储)

Others

How can I fix that?

Originally created by @mrgaolei on GitHub (Jan 4, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1782 Originally assigned to: @ouqingliang on GitHub. ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info kt-kernel 0.5.0.post1 CPU: 2 x 6416h RAM: 1T DDR5 GPU: 4090 48G Ubuntu 24.04 CUDA 13.0 ### Reproduction convert script ```bash n scripts/convert_cpu_weights.py -i /opt/ai-models/DeepSeek-V3.2 --input-type fp8 --output /opt/ai-models/DeepSeek-V3.2-INT4 --quant-method int4 --cpuinfer-threads 10 --threadpool-count 1 --no-merge-safetensor``` but throw this error: ```bash Layer 59: 256 experts (0-255) Layer 60: 256 experts (0-255) Layer 61: 256 experts (0-255) Processing layer 3 (1/59)... Converting layer 3 with 256 experts via online quantization... Loaded weights shapes: gate_proj: torch.Size([256, 2048, 7168]) up_proj: torch.Size([256, 2048, 7168]) down_proj: torch.Size([256, 7168, 2048]) CPUInfer[0x43564070]: Hello WorkerPool[0x43625890] 1 subpools, [numa:threads][0:10] ===========In NumaPool============ In Numa Worker Pool at NUMA 0, 10 threads TP MOE layer 3, pool: 0x43625890, expert num: 256, num_experts_per_tok: 8 From BF16 online quant from bf16 段错误 (核心已转储) ``` ### Others How can I fix that?
bsakaguchi commented 2026-10-29 14:48:47 +00:00
Author
Owner
Copy link

@ouqingliang commented on GitHub (Jan 4, 2026):

This bug has already been fixed. You can try again using the latest code on the main branch.

<!-- gh-comment-id:3708005727 --> @ouqingliang commented on GitHub (Jan 4, 2026): This bug has already been fixed. You can try again using the latest code on the main branch.
bsakaguchi commented 2026-10-29 14:48:47 +00:00
Author
Owner
Copy link

@pchela2305 commented on GitHub (Jan 4, 2026):

the error has not gone away

System Info

kt-kernel 0.5.0.post1
CPU: G2SR+
RAM: 512T DDR5
GPU: 4090 24G

Ubuntu 24.04
CUDA 13.0

CUDA_VISIBLE_DEVICES=0 python kt-kernel/scripts/convert_cpu_weights.py --input-path /home/llm/models/DeepSeek-V3.2 --input-type fp8   --output /home/ssl90/models/DeepSeek-V3.2-INT4 --quant-method int4 --cpuinfer-threads 64 --no-merge-safetensor --threadpool-count 1
  
   ...
  Layer 52: 256 experts (0-255)
  Layer 53: 256 experts (0-255)
  Layer 54: 256 experts (0-255)
  Layer 55: 256 experts (0-255)
  Layer 56: 256 experts (0-255)
  Layer 57: 256 experts (0-255)
  Layer 58: 256 experts (0-255)
  Layer 59: 256 experts (0-255)
  Layer 60: 256 experts (0-255)
  Layer 61: 256 experts (0-255)
Processing layer 3 (1/59)...
Converting layer 3 with 256 experts via online quantization...
  Loaded weights shapes:
    gate_proj: torch.Size([256, 2048, 7168])
    up_proj: torch.Size([256, 2048, 7168])
    down_proj: torch.Size([256, 7168, 2048])
CPUInfer[0xd17d010]: Hello
WorkerPool[0xd228560] 1 subpools, [numa:threads][0:64]
===========In NumaPool============
In Numa Worker Pool at NUMA 0, 64 threads
TP MOE layer 3, pool: 0xd228560, expert num: 256, num_experts_per_tok: 8
Creating AMX_MOE_TP 0 at numa 0
Creating "/home/llm/models/DeepSeek-V3.2-INT4/_layer_3/_numa_0"
From BF16
  online quant from bf16
  Layer 3 quantized and saved in 33.81s
  Keeping layer folder structure at /home/llm/models/DeepSeek-V3.2-INT4/_layer_3
Segmentation fault (memory image has been flushed to disk)
<!-- gh-comment-id:3708170427 --> @pchela2305 commented on GitHub (Jan 4, 2026): the error has not gone away ### System Info kt-kernel 0.5.0.post1 CPU: G2SR+ RAM: 512T DDR5 GPU: 4090 24G Ubuntu 24.04 CUDA 13.0 ```bash CUDA_VISIBLE_DEVICES=0 python kt-kernel/scripts/convert_cpu_weights.py --input-path /home/llm/models/DeepSeek-V3.2 --input-type fp8 --output /home/ssl90/models/DeepSeek-V3.2-INT4 --quant-method int4 --cpuinfer-threads 64 --no-merge-safetensor --threadpool-count 1 ... Layer 52: 256 experts (0-255) Layer 53: 256 experts (0-255) Layer 54: 256 experts (0-255) Layer 55: 256 experts (0-255) Layer 56: 256 experts (0-255) Layer 57: 256 experts (0-255) Layer 58: 256 experts (0-255) Layer 59: 256 experts (0-255) Layer 60: 256 experts (0-255) Layer 61: 256 experts (0-255) Processing layer 3 (1/59)... Converting layer 3 with 256 experts via online quantization... Loaded weights shapes: gate_proj: torch.Size([256, 2048, 7168]) up_proj: torch.Size([256, 2048, 7168]) down_proj: torch.Size([256, 7168, 2048]) CPUInfer[0xd17d010]: Hello WorkerPool[0xd228560] 1 subpools, [numa:threads][0:64] ===========In NumaPool============ In Numa Worker Pool at NUMA 0, 64 threads TP MOE layer 3, pool: 0xd228560, expert num: 256, num_experts_per_tok: 8 Creating AMX_MOE_TP 0 at numa 0 Creating "/home/llm/models/DeepSeek-V3.2-INT4/_layer_3/_numa_0" From BF16 online quant from bf16 Layer 3 quantized and saved in 33.81s Keeping layer folder structure at /home/llm/models/DeepSeek-V3.2-INT4/_layer_3 Segmentation fault (memory image has been flushed to disk) ```
bsakaguchi commented 2026-10-29 14:48:47 +00:00
Author
Owner
Copy link

@mrgaolei commented on GitHub (Jan 5, 2026):

This bug has already been fixed. You can try again using the latest code on the main branch.

No!, I pull the branch main laest code, the error even happen.
But I reverse to tag v0.4.4 it run success.
I think the bug has not fixed.

<!-- gh-comment-id:3708739938 --> @mrgaolei commented on GitHub (Jan 5, 2026): > This bug has already been fixed. You can try again using the latest code on the main branch. No!, I pull the branch `main` laest code, the error even happen. But I reverse to tag `v0.4.4` it run success. I think the bug has not fixed.
bsakaguchi commented 2026-10-29 14:48:47 +00:00
Author
Owner
Copy link

@pchela2305 commented on GitHub (Jan 6, 2026):

cuda 13.1 + build NCCL + git pull + rebuild kt-kernel = work

NCCL- is not sure if it is needed :)

<!-- gh-comment-id:3716662048 --> @pchela2305 commented on GitHub (Jan 6, 2026): cuda 13.1 + build NCCL + git pull + rebuild kt-kernel = work NCCL- is not sure if it is needed :)
bsakaguchi commented 2026-10-29 14:48:47 +00:00
Author
Owner
Copy link

@ouqingliang commented on GitHub (Jan 7, 2026):

This bug has already been fixed. You can try again using the latest code on the main branch.

No!, I pull the branch main laest code, the error even happen. But I reverse to tag v0.4.4 it run success. I think the bug has not fixed.

Thank you very much for your reply. You’re right that the problem was not fully resolved in the previous main branch. I delete the gate_proj_ member in moe.hpp, which may causes segmentation fault during the conversion process.

I’m now able to successfully run the V3.2 conversion with the latest main branch. Sorry for the confusion earlier.

<!-- gh-comment-id:3717533673 --> @ouqingliang commented on GitHub (Jan 7, 2026): > > This bug has already been fixed. You can try again using the latest code on the main branch. > > No!, I pull the branch `main` laest code, the error even happen. But I reverse to tag `v0.4.4` it run success. I think the bug has not fixed. Thank you very much for your reply. You’re right that the problem was not fully resolved in the previous main branch. I delete the `gate_proj_` member in `moe.hpp`, which may causes segmentation fault during the conversion process. I’m now able to successfully run the V3.2 conversion with the latest main branch. Sorry for the confusion earlier.
bsakaguchi commented 2026-10-29 14:48:48 +00:00
Author
Owner
Copy link

@Jiqiren87 commented on GitHub (Jan 13, 2026):

目前该问题得到解决了么？我还遇到了可能是类似的问题。

System Info
kt-kernel 0.5.0.post1
CPU: 8460*2
RAM: 2T DDR5
GPU: L40s 48G

Ubuntu 22.04
CUDA 12.8

我使用
kt quant -m int8 -o '/home/netadmin/models/DeepSeek-V3.2-INT8' --cpu-threads 96 --numa-nodes 2 '/home/netadmin/models/DeepSeek-V3.2'

或是

到/scripts中执行
python convert_cpu_weights.py --input-path /home/netadmin/models/DeepSeek-V3.2 --input-type fp8 --output /home/netadmin/models/DeepSeek-V3.2-INT8 --quant-method int8 --cpuinfer-threads 96 --threadpool-count 2

都会出现类似错误：

前者为：

Loading model configuration...
FP8 quantization config detected:
format: e4m3
weight_block_size: [128, 128]
Model config: {'num_experts': 256, 'num_experts_per_tok': 8, 'hidden_size': 7168, 'moe_intermediate_size': 2048, 'fp8_weight_block_size': [128, 128]}
num_experts: 256
num_experts_per_tok: 8
hidden_size: 7168
moe_intermediate_size: 2048
CPU inference config:
cpuinfer_threads: 96
threadpool_count: 2

Loading safetensors files from /home/netadmin/models/DeepSeek-V3.2
Loaded: model-00001-of-000163.safetensors (290 tensors)
Loaded: model-00002-of-000163.safetensors (586 tensors)
Loaded: model-00003-of-000163.safetensors (586 tensors)
Loaded: model-00004-of-000163.safetensors (581 tensors)
Loaded: model-00005-of-000163.safetensors (586 tensors)
Loaded: model-00006-of-000163.safetensors (587 tensors)
Loaded: model-00007-of-000163.safetensors (582 tensors)
Loaded: model-00008-of-000163.safetensors (586 tensors)
Loaded: model-00009-of-000163.safetensors (581 tensors)
Loaded: model-00010-of-000163.safetensors (586 tensors)
Loaded: model-00011-of-000163.safetensors (586 tensors)
Loaded: model-00012-of-000163.safetensors (205 tensors)
Loaded: model-00013-of-000163.safetensors (578 tensors)
Loaded: model-00014-of-000163.safetensors (586 tensors)
Loaded: model-00015-of-000163.safetensors (581 tensors)
Loaded: model-00016-of-000163.safetensors (586 tensors)
Loaded: model-00017-of-000163.safetensors (586 tensors)
Loaded: model-00018-of-000163.safetensors (581 tensors)
Loaded: model-00019-of-000163.safetensors (586 tensors)
Loaded: model-00020-of-000163.safetensors (586 tensors)
Loaded: model-00021-of-000163.safetensors (581 tensors)
Loaded: model-00022-of-000163.safetensors (586 tensors)
Loaded: model-00023-of-000163.safetensors (581 tensors)
Loaded: model-00024-of-000163.safetensors (586 tensors)
Loaded: model-00025-of-000163.safetensors (586 tensors)
Loaded: model-00026-of-000163.safetensors (581 tensors)
Loaded: model-00027-of-000163.safetensors (586 tensors)
Loaded: model-00028-of-000163.safetensors (586 tensors)
Loaded: model-00029-of-000163.safetensors (581 tensors)
Loaded: model-00030-of-000163.safetensors (586 tensors)
Loaded: model-00031-of-000163.safetensors (581 tensors)
Loaded: model-00032-of-000163.safetensors (586 tensors)
Loaded: model-00033-of-000163.safetensors (586 tensors)
Loaded: model-00034-of-000163.safetensors (257 tensors)
Loaded: model-00035-of-000163.safetensors (578 tensors)
Loaded: model-00036-of-000163.safetensors (586 tensors)
Loaded: model-00037-of-000163.safetensors (581 tensors)
Loaded: model-00038-of-000163.safetensors (586 tensors)
Loaded: model-00039-of-000163.safetensors (586 tensors)
Loaded: model-00040-of-000163.safetensors (581 tensors)
Loaded: model-00041-of-000163.safetensors (586 tensors)
Loaded: model-00042-of-000163.safetensors (586 tensors)
Loaded: model-00043-of-000163.safetensors (581 tensors)
Loaded: model-00044-of-000163.safetensors (586 tensors)
Loaded: model-00045-of-000163.safetensors (581 tensors)
Loaded: model-00046-of-000163.safetensors (586 tensors)
Loaded: model-00047-of-000163.safetensors (586 tensors)
Loaded: model-00048-of-000163.safetensors (581 tensors)
Loaded: model-00049-of-000163.safetensors (586 tensors)
Loaded: model-00050-of-000163.safetensors (586 tensors)
Loaded: model-00051-of-000163.safetensors (581 tensors)
Loaded: model-00052-of-000163.safetensors (586 tensors)
Loaded: model-00053-of-000163.safetensors (581 tensors)
Loaded: model-00054-of-000163.safetensors (586 tensors)
Loaded: model-00055-of-000163.safetensors (586 tensors)
Loaded: model-00056-of-000163.safetensors (257 tensors)
Loaded: model-00057-of-000163.safetensors (578 tensors)
Loaded: model-00058-of-000163.safetensors (586 tensors)
Loaded: model-00059-of-000163.safetensors (581 tensors)
Loaded: model-00060-of-000163.safetensors (586 tensors)
Loaded: model-00061-of-000163.safetensors (586 tensors)
Loaded: model-00062-of-000163.safetensors (581 tensors)
Loaded: model-00063-of-000163.safetensors (586 tensors)
Loaded: model-00064-of-000163.safetensors (586 tensors)
Loaded: model-00065-of-000163.safetensors (581 tensors)
Loaded: model-00066-of-000163.safetensors (586 tensors)
Loaded: model-00067-of-000163.safetensors (581 tensors)
Loaded: model-00068-of-000163.safetensors (586 tensors)
Loaded: model-00069-of-000163.safetensors (586 tensors)
Loaded: model-00070-of-000163.safetensors (581 tensors)
Loaded: model-00071-of-000163.safetensors (586 tensors)
Loaded: model-00072-of-000163.safetensors (586 tensors)
Loaded: model-00073-of-000163.safetensors (581 tensors)
Loaded: model-00074-of-000163.safetensors (586 tensors)
Loaded: model-00075-of-000163.safetensors (581 tensors)
Loaded: model-00076-of-000163.safetensors (586 tensors)
Loaded: model-00077-of-000163.safetensors (586 tensors)
Loaded: model-00078-of-000163.safetensors (257 tensors)
Loaded: model-00079-of-000163.safetensors (578 tensors)
Loaded: model-00080-of-000163.safetensors (586 tensors)
Loaded: model-00081-of-000163.safetensors (581 tensors)
Loaded: model-00082-of-000163.safetensors (586 tensors)
Loaded: model-00083-of-000163.safetensors (586 tensors)
Loaded: model-00084-of-000163.safetensors (581 tensors)
Loaded: model-00085-of-000163.safetensors (586 tensors)
Loaded: model-00086-of-000163.safetensors (586 tensors)
Loaded: model-00087-of-000163.safetensors (581 tensors)
Loaded: model-00088-of-000163.safetensors (586 tensors)
Loaded: model-00089-of-000163.safetensors (581 tensors)
Loaded: model-00090-of-000163.safetensors (586 tensors)
Loaded: model-00091-of-000163.safetensors (586 tensors)
Loaded: model-00092-of-000163.safetensors (581 tensors)
Loaded: model-00093-of-000163.safetensors (586 tensors)
Loaded: model-00094-of-000163.safetensors (586 tensors)
Loaded: model-00095-of-000163.safetensors (581 tensors)
Loaded: model-00096-of-000163.safetensors (586 tensors)
Loaded: model-00097-of-000163.safetensors (581 tensors)
Loaded: model-00098-of-000163.safetensors (586 tensors)
Loaded: model-00099-of-000163.safetensors (586 tensors)
Loaded: model-00100-of-000163.safetensors (257 tensors)
Loaded: model-00101-of-000163.safetensors (578 tensors)
Loaded: model-00102-of-000163.safetensors (586 tensors)
Loaded: model-00103-of-000163.safetensors (581 tensors)
Loaded: model-00104-of-000163.safetensors (586 tensors)
Loaded: model-00105-of-000163.safetensors (586 tensors)
Loaded: model-00106-of-000163.safetensors (581 tensors)
Loaded: model-00107-of-000163.safetensors (586 tensors)
Loaded: model-00108-of-000163.safetensors (586 tensors)
Loaded: model-00109-of-000163.safetensors (581 tensors)
Loaded: model-00110-of-000163.safetensors (586 tensors)
Loaded: model-00111-of-000163.safetensors (581 tensors)
Loaded: model-00112-of-000163.safetensors (586 tensors)
Loaded: model-00113-of-000163.safetensors (586 tensors)
Loaded: model-00114-of-000163.safetensors (581 tensors)
Loaded: model-00115-of-000163.safetensors (586 tensors)
Loaded: model-00116-of-000163.safetensors (586 tensors)
Loaded: model-00117-of-000163.safetensors (581 tensors)
Loaded: model-00118-of-000163.safetensors (586 tensors)
Loaded: model-00119-of-000163.safetensors (581 tensors)
Loaded: model-00120-of-000163.safetensors (586 tensors)
Loaded: model-00121-of-000163.safetensors (586 tensors)
Loaded: model-00122-of-000163.safetensors (257 tensors)
Loaded: model-00123-of-000163.safetensors (578 tensors)
Loaded: model-00124-of-000163.safetensors (586 tensors)
Loaded: model-00125-of-000163.safetensors (581 tensors)
Loaded: model-00126-of-000163.safetensors (586 tensors)
Loaded: model-00127-of-000163.safetensors (586 tensors)
Loaded: model-00128-of-000163.safetensors (581 tensors)
Loaded: model-00129-of-000163.safetensors (586 tensors)
Loaded: model-00130-of-000163.safetensors (586 tensors)
Loaded: model-00131-of-000163.safetensors (581 tensors)
Loaded: model-00132-of-000163.safetensors (586 tensors)
Loaded: model-00133-of-000163.safetensors (581 tensors)
Loaded: model-00134-of-000163.safetensors (586 tensors)
Loaded: model-00135-of-000163.safetensors (586 tensors)
Loaded: model-00136-of-000163.safetensors (581 tensors)
Loaded: model-00137-of-000163.safetensors (586 tensors)
Loaded: model-00138-of-000163.safetensors (586 tensors)
Loaded: model-00139-of-000163.safetensors (581 tensors)
Loaded: model-00140-of-000163.safetensors (586 tensors)
Loaded: model-00141-of-000163.safetensors (445 tensors)
Loaded: model-00142-of-000163.safetensors (578 tensors)
Loaded: model-00143-of-000163.safetensors (586 tensors)
Loaded: model-00144-of-000163.safetensors (581 tensors)
Loaded: model-00145-of-000163.safetensors (586 tensors)
Loaded: model-00146-of-000163.safetensors (586 tensors)
Loaded: model-00147-of-000163.safetensors (581 tensors)
Loaded: model-00148-of-000163.safetensors (586 tensors)
Loaded: model-00149-of-000163.safetensors (586 tensors)
Loaded: model-00150-of-000163.safetensors (581 tensors)
Loaded: model-00151-of-000163.safetensors (586 tensors)
Loaded: model-00152-of-000163.safetensors (581 tensors)
Loaded: model-00153-of-000163.safetensors (586 tensors)
Loaded: model-00154-of-000163.safetensors (586 tensors)
Loaded: model-00155-of-000163.safetensors (581 tensors)
Loaded: model-00156-of-000163.safetensors (586 tensors)
Loaded: model-00157-of-000163.safetensors (586 tensors)
Loaded: model-00158-of-000163.safetensors (581 tensors)
Loaded: model-00159-of-000163.safetensors (586 tensors)
Loaded: model-00160-of-000163.safetensors (458 tensors)
Loaded: model-00161-of-000163.safetensors (593 tensors)
Loaded: model-00162-of-000163.safetensors (586 tensors)
Loaded: model-00163-of-000163.safetensors (381 tensors)
Total tensors loaded: 92425
Starting conversion...
Input: /home/netadmin/models/DeepSeek-V3.2
Output: /home/netadmin/models/DeepSeek-V3.2-INT8
Found 59 layers with MoE experts:
Layer 3: 256 experts (0-255)
Layer 4: 256 experts (0-255)
Layer 5: 256 experts (0-255)
Layer 6: 256 experts (0-255)
Layer 7: 256 experts (0-255)
Layer 8: 256 experts (0-255)
Layer 9: 256 experts (0-255)
Layer 10: 256 experts (0-255)
Layer 11: 256 experts (0-255)
Layer 12: 256 experts (0-255)
Layer 13: 256 experts (0-255)
Layer 14: 256 experts (0-255)
Layer 15: 256 experts (0-255)
Layer 16: 256 experts (0-255)
Layer 17: 256 experts (0-255)
Layer 18: 256 experts (0-255)
Layer 19: 256 experts (0-255)
Layer 20: 256 experts (0-255)
Layer 21: 256 experts (0-255)
Layer 22: 256 experts (0-255)
Layer 23: 256 experts (0-255)
Layer 24: 256 experts (0-255)
Layer 25: 256 experts (0-255)
Layer 26: 256 experts (0-255)
Layer 27: 256 experts (0-255)
Layer 28: 256 experts (0-255)
Layer 29: 256 experts (0-255)
Layer 30: 256 experts (0-255)
Layer 31: 256 experts (0-255)
Layer 32: 256 experts (0-255)
Layer 33: 256 experts (0-255)
Layer 34: 256 experts (0-255)
Layer 35: 256 experts (0-255)
Layer 36: 256 experts (0-255)
Layer 37: 256 experts (0-255)
Layer 38: 256 experts (0-255)
Layer 39: 256 experts (0-255)
Layer 40: 256 experts (0-255)
Layer 41: 256 experts (0-255)
Layer 42: 256 experts (0-255)
Layer 43: 256 experts (0-255)
Layer 44: 256 experts (0-255)
Layer 45: 256 experts (0-255)
Layer 46: 256 experts (0-255)
Layer 47: 256 experts (0-255)
Layer 48: 256 experts (0-255)
Layer 49: 256 experts (0-255)
Layer 50: 256 experts (0-255)
Layer 51: 256 experts (0-255)
Layer 52: 256 experts (0-255)
Layer 53: 256 experts (0-255)
Layer 54: 256 experts (0-255)
Layer 55: 256 experts (0-255)
Layer 56: 256 experts (0-255)
Layer 57: 256 experts (0-255)
Layer 58: 256 experts (0-255)
Layer 59: 256 experts (0-255)
Layer 60: 256 experts (0-255)
Layer 61: 256 experts (0-255)
Processing layer 3 (1/59)...
Converting layer 3 with 256 experts via online quantization...
Loaded weights shapes:
gate_proj: torch.Size([256, 2048, 7168])
up_proj: torch.Size([256, 2048, 7168])
down_proj: torch.Size([256, 7168, 2048])
CPUInfer[0x4394d820]: Hello
WorkerPool[0x43a02ff0] 2 subpools, [numa:threads][0:48] [1:48]
===========In NumaPool============
In Numa Worker Pool at NUMA 0, 48 threads
===========In NumaPool============
In Numa Worker Pool at NUMA 1, 48 threads
TP MOE layer 3, pool: 0x43a02ff0, expert num: 256, num_experts_per_tok: 8
From BF16
online quant from bf16
terminate called after throwing an instance of 'std::logic_error'
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
what(): basic_string::_M_construct null not valid
terminate called recursively
✗ Quantization failed with exit code -6

后者为

python convert_cpu_weights.py --input-path /home/netadmin/models/DeepSeek-V3.2 --input-type fp8 --output /home/netadmin/models/DeepSeek-V3.2-INT8 --quant-method int8 --cpuinfer-threads 96 --threadpool-count 2
Loading model configuration...
FP8 quantization config detected:
format: e4m3
weight_block_size: [128, 128]
Model config: {'num_experts': 256, 'num_experts_per_tok': 8, 'hidden_size': 7168, 'moe_intermediate_size': 2048, 'fp8_weight_block_size': [128, 128]}
num_experts: 256
num_experts_per_tok: 8
hidden_size: 7168
moe_intermediate_size: 2048
CPU inference config:
cpuinfer_threads: 96
threadpool_count: 2

Loading safetensors files from /home/netadmin/models/DeepSeek-V3.2
Loaded: model-00001-of-000163.safetensors (290 tensors)
Loaded: model-00002-of-000163.safetensors (586 tensors)
Loaded: model-00003-of-000163.safetensors (586 tensors)
Loaded: model-00004-of-000163.safetensors (581 tensors)
Loaded: model-00005-of-000163.safetensors (586 tensors)
Loaded: model-00006-of-000163.safetensors (587 tensors)
Loaded: model-00007-of-000163.safetensors (582 tensors)
Loaded: model-00008-of-000163.safetensors (586 tensors)
Loaded: model-00009-of-000163.safetensors (581 tensors)
Loaded: model-00010-of-000163.safetensors (586 tensors)
Loaded: model-00011-of-000163.safetensors (586 tensors)
Loaded: model-00012-of-000163.safetensors (205 tensors)
Loaded: model-00013-of-000163.safetensors (578 tensors)
Loaded: model-00014-of-000163.safetensors (586 tensors)
Loaded: model-00015-of-000163.safetensors (581 tensors)
Loaded: model-00016-of-000163.safetensors (586 tensors)
Loaded: model-00017-of-000163.safetensors (586 tensors)
Loaded: model-00018-of-000163.safetensors (581 tensors)
Loaded: model-00019-of-000163.safetensors (586 tensors)
Loaded: model-00020-of-000163.safetensors (586 tensors)
Loaded: model-00021-of-000163.safetensors (581 tensors)
Loaded: model-00022-of-000163.safetensors (586 tensors)
Loaded: model-00023-of-000163.safetensors (581 tensors)
Loaded: model-00024-of-000163.safetensors (586 tensors)
Loaded: model-00025-of-000163.safetensors (586 tensors)
Loaded: model-00026-of-000163.safetensors (581 tensors)
Loaded: model-00027-of-000163.safetensors (586 tensors)
Loaded: model-00028-of-000163.safetensors (586 tensors)
Loaded: model-00029-of-000163.safetensors (581 tensors)
Loaded: model-00030-of-000163.safetensors (586 tensors)
Loaded: model-00031-of-000163.safetensors (581 tensors)
Loaded: model-00032-of-000163.safetensors (586 tensors)
Loaded: model-00033-of-000163.safetensors (586 tensors)
Loaded: model-00034-of-000163.safetensors (257 tensors)
Loaded: model-00035-of-000163.safetensors (578 tensors)
Loaded: model-00036-of-000163.safetensors (586 tensors)
Loaded: model-00037-of-000163.safetensors (581 tensors)
Loaded: model-00038-of-000163.safetensors (586 tensors)
Loaded: model-00039-of-000163.safetensors (586 tensors)
Loaded: model-00040-of-000163.safetensors (581 tensors)
Loaded: model-00041-of-000163.safetensors (586 tensors)
Loaded: model-00042-of-000163.safetensors (586 tensors)
Loaded: model-00043-of-000163.safetensors (581 tensors)
Loaded: model-00044-of-000163.safetensors (586 tensors)
Loaded: model-00045-of-000163.safetensors (581 tensors)
Loaded: model-00046-of-000163.safetensors (586 tensors)
Loaded: model-00047-of-000163.safetensors (586 tensors)
Loaded: model-00048-of-000163.safetensors (581 tensors)
Loaded: model-00049-of-000163.safetensors (586 tensors)
Loaded: model-00050-of-000163.safetensors (586 tensors)
Loaded: model-00051-of-000163.safetensors (581 tensors)
Loaded: model-00052-of-000163.safetensors (586 tensors)
Loaded: model-00053-of-000163.safetensors (581 tensors)
Loaded: model-00054-of-000163.safetensors (586 tensors)
Loaded: model-00055-of-000163.safetensors (586 tensors)
Loaded: model-00056-of-000163.safetensors (257 tensors)
Loaded: model-00057-of-000163.safetensors (578 tensors)
Loaded: model-00058-of-000163.safetensors (586 tensors)
Loaded: model-00059-of-000163.safetensors (581 tensors)
Loaded: model-00060-of-000163.safetensors (586 tensors)
Loaded: model-00061-of-000163.safetensors (586 tensors)
Loaded: model-00062-of-000163.safetensors (581 tensors)
Loaded: model-00063-of-000163.safetensors (586 tensors)
Loaded: model-00064-of-000163.safetensors (586 tensors)
Loaded: model-00065-of-000163.safetensors (581 tensors)
Loaded: model-00066-of-000163.safetensors (586 tensors)
Loaded: model-00067-of-000163.safetensors (581 tensors)
Loaded: model-00068-of-000163.safetensors (586 tensors)
Loaded: model-00069-of-000163.safetensors (586 tensors)
Loaded: model-00070-of-000163.safetensors (581 tensors)
Loaded: model-00071-of-000163.safetensors (586 tensors)
Loaded: model-00072-of-000163.safetensors (586 tensors)
Loaded: model-00073-of-000163.safetensors (581 tensors)
Loaded: model-00074-of-000163.safetensors (586 tensors)
Loaded: model-00075-of-000163.safetensors (581 tensors)
Loaded: model-00076-of-000163.safetensors (586 tensors)
Loaded: model-00077-of-000163.safetensors (586 tensors)
Loaded: model-00078-of-000163.safetensors (257 tensors)
Loaded: model-00079-of-000163.safetensors (578 tensors)
Loaded: model-00080-of-000163.safetensors (586 tensors)
Loaded: model-00081-of-000163.safetensors (581 tensors)
Loaded: model-00082-of-000163.safetensors (586 tensors)
Loaded: model-00083-of-000163.safetensors (586 tensors)
Loaded: model-00084-of-000163.safetensors (581 tensors)
Loaded: model-00085-of-000163.safetensors (586 tensors)
Loaded: model-00086-of-000163.safetensors (586 tensors)
Loaded: model-00087-of-000163.safetensors (581 tensors)
Loaded: model-00088-of-000163.safetensors (586 tensors)
Loaded: model-00089-of-000163.safetensors (581 tensors)
Loaded: model-00090-of-000163.safetensors (586 tensors)
Loaded: model-00091-of-000163.safetensors (586 tensors)
Loaded: model-00092-of-000163.safetensors (581 tensors)
Loaded: model-00093-of-000163.safetensors (586 tensors)
Loaded: model-00094-of-000163.safetensors (586 tensors)
Loaded: model-00095-of-000163.safetensors (581 tensors)
Loaded: model-00096-of-000163.safetensors (586 tensors)
Loaded: model-00097-of-000163.safetensors (581 tensors)
Loaded: model-00098-of-000163.safetensors (586 tensors)
Loaded: model-00099-of-000163.safetensors (586 tensors)
Loaded: model-00100-of-000163.safetensors (257 tensors)
Loaded: model-00101-of-000163.safetensors (578 tensors)
Loaded: model-00102-of-000163.safetensors (586 tensors)
Loaded: model-00103-of-000163.safetensors (581 tensors)
Loaded: model-00104-of-000163.safetensors (586 tensors)
Loaded: model-00105-of-000163.safetensors (586 tensors)
Loaded: model-00106-of-000163.safetensors (581 tensors)
Loaded: model-00107-of-000163.safetensors (586 tensors)
Loaded: model-00108-of-000163.safetensors (586 tensors)
Loaded: model-00109-of-000163.safetensors (581 tensors)
Loaded: model-00110-of-000163.safetensors (586 tensors)
Loaded: model-00111-of-000163.safetensors (581 tensors)
Loaded: model-00112-of-000163.safetensors (586 tensors)
Loaded: model-00113-of-000163.safetensors (586 tensors)
Loaded: model-00114-of-000163.safetensors (581 tensors)
Loaded: model-00115-of-000163.safetensors (586 tensors)
Loaded: model-00116-of-000163.safetensors (586 tensors)
Loaded: model-00117-of-000163.safetensors (581 tensors)
Loaded: model-00118-of-000163.safetensors (586 tensors)
Loaded: model-00119-of-000163.safetensors (581 tensors)
Loaded: model-00120-of-000163.safetensors (586 tensors)
Loaded: model-00121-of-000163.safetensors (586 tensors)
Loaded: model-00122-of-000163.safetensors (257 tensors)
Loaded: model-00123-of-000163.safetensors (578 tensors)
Loaded: model-00124-of-000163.safetensors (586 tensors)
Loaded: model-00125-of-000163.safetensors (581 tensors)
Loaded: model-00126-of-000163.safetensors (586 tensors)
Loaded: model-00127-of-000163.safetensors (586 tensors)
Loaded: model-00128-of-000163.safetensors (581 tensors)
Loaded: model-00129-of-000163.safetensors (586 tensors)
Loaded: model-00130-of-000163.safetensors (586 tensors)
Loaded: model-00131-of-000163.safetensors (581 tensors)
Loaded: model-00132-of-000163.safetensors (586 tensors)
Loaded: model-00133-of-000163.safetensors (581 tensors)
Loaded: model-00134-of-000163.safetensors (586 tensors)
Loaded: model-00135-of-000163.safetensors (586 tensors)
Loaded: model-00136-of-000163.safetensors (581 tensors)
Loaded: model-00137-of-000163.safetensors (586 tensors)
Loaded: model-00138-of-000163.safetensors (586 tensors)
Loaded: model-00139-of-000163.safetensors (581 tensors)
Loaded: model-00140-of-000163.safetensors (586 tensors)
Loaded: model-00141-of-000163.safetensors (445 tensors)
Loaded: model-00142-of-000163.safetensors (578 tensors)
Loaded: model-00143-of-000163.safetensors (586 tensors)
Loaded: model-00144-of-000163.safetensors (581 tensors)
Loaded: model-00145-of-000163.safetensors (586 tensors)
Loaded: model-00146-of-000163.safetensors (586 tensors)
Loaded: model-00147-of-000163.safetensors (581 tensors)
Loaded: model-00148-of-000163.safetensors (586 tensors)
Loaded: model-00149-of-000163.safetensors (586 tensors)
Loaded: model-00150-of-000163.safetensors (581 tensors)
Loaded: model-00151-of-000163.safetensors (586 tensors)
Loaded: model-00152-of-000163.safetensors (581 tensors)
Loaded: model-00153-of-000163.safetensors (586 tensors)
Loaded: model-00154-of-000163.safetensors (586 tensors)
Loaded: model-00155-of-000163.safetensors (581 tensors)
Loaded: model-00156-of-000163.safetensors (586 tensors)
Loaded: model-00157-of-000163.safetensors (586 tensors)
Loaded: model-00158-of-000163.safetensors (581 tensors)
Loaded: model-00159-of-000163.safetensors (586 tensors)
Loaded: model-00160-of-000163.safetensors (458 tensors)
Loaded: model-00161-of-000163.safetensors (593 tensors)
Loaded: model-00162-of-000163.safetensors (586 tensors)
Loaded: model-00163-of-000163.safetensors (381 tensors)
Total tensors loaded: 92425
Starting conversion...
Input: /home/netadmin/models/DeepSeek-V3.2
Output: /home/netadmin/models/DeepSeek-V3.2-INT8
Found 59 layers with MoE experts:
Layer 3: 256 experts (0-255)
Layer 4: 256 experts (0-255)
Layer 5: 256 experts (0-255)
Layer 6: 256 experts (0-255)
Layer 7: 256 experts (0-255)
Layer 8: 256 experts (0-255)
Layer 9: 256 experts (0-255)
Layer 10: 256 experts (0-255)
Layer 11: 256 experts (0-255)
Layer 12: 256 experts (0-255)
Layer 13: 256 experts (0-255)
Layer 14: 256 experts (0-255)
Layer 15: 256 experts (0-255)
Layer 16: 256 experts (0-255)
Layer 17: 256 experts (0-255)
Layer 18: 256 experts (0-255)
Layer 19: 256 experts (0-255)
Layer 20: 256 experts (0-255)
Layer 21: 256 experts (0-255)
Layer 22: 256 experts (0-255)
Layer 23: 256 experts (0-255)
Layer 24: 256 experts (0-255)
Layer 25: 256 experts (0-255)
Layer 26: 256 experts (0-255)
Layer 27: 256 experts (0-255)
Layer 28: 256 experts (0-255)
Layer 29: 256 experts (0-255)
Layer 30: 256 experts (0-255)
Layer 31: 256 experts (0-255)
Layer 32: 256 experts (0-255)
Layer 33: 256 experts (0-255)
Layer 34: 256 experts (0-255)
Layer 35: 256 experts (0-255)
Layer 36: 256 experts (0-255)
Layer 37: 256 experts (0-255)
Layer 38: 256 experts (0-255)
Layer 39: 256 experts (0-255)
Layer 40: 256 experts (0-255)
Layer 41: 256 experts (0-255)
Layer 42: 256 experts (0-255)
Layer 43: 256 experts (0-255)
Layer 44: 256 experts (0-255)
Layer 45: 256 experts (0-255)
Layer 46: 256 experts (0-255)
Layer 47: 256 experts (0-255)
Layer 48: 256 experts (0-255)
Layer 49: 256 experts (0-255)
Layer 50: 256 experts (0-255)
Layer 51: 256 experts (0-255)
Layer 52: 256 experts (0-255)
Layer 53: 256 experts (0-255)
Layer 54: 256 experts (0-255)
Layer 55: 256 experts (0-255)
Layer 56: 256 experts (0-255)
Layer 57: 256 experts (0-255)
Layer 58: 256 experts (0-255)
Layer 59: 256 experts (0-255)
Layer 60: 256 experts (0-255)
Layer 61: 256 experts (0-255)
Processing layer 3 (1/59)...
Converting layer 3 with 256 experts via online quantization...
Loaded weights shapes:
gate_proj: torch.Size([256, 2048, 7168])
up_proj: torch.Size([256, 2048, 7168])
down_proj: torch.Size([256, 7168, 2048])
CPUInfer[0x154dfd80]: Hello
WorkerPool[0x154dfd10] 2 subpools, [numa:threads][0:48] [1:48]
===========In NumaPool============
In Numa Worker Pool at NUMA 0, 48 threads
===========In NumaPool============
In Numa Worker Pool at NUMA 1, 48 threads
TP MOE layer 3, pool: 0x154dfd10, expert num: 256, num_experts_per_tok: 8
From BF16
online quant from bf16
terminate called after throwing an instance of 'std::logic_error'
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
已中止 (核心已转储)

<!-- gh-comment-id:3743093094 --> @Jiqiren87 commented on GitHub (Jan 13, 2026): 目前该问题得到解决了么？我还遇到了可能是类似的问题。 System Info kt-kernel 0.5.0.post1 CPU: 8460*2 RAM: 2T DDR5 GPU: L40s 48G Ubuntu 22.04 CUDA 12.8 我使用 kt quant -m int8 -o '/home/netadmin/models/DeepSeek-V3.2-INT8' --cpu-threads 96 --numa-nodes 2 '/home/netadmin/models/DeepSeek-V3.2' 或是 到/scripts中执行 python convert_cpu_weights.py --input-path /home/netadmin/models/DeepSeek-V3.2 --input-type fp8 --output /home/netadmin/models/DeepSeek-V3.2-INT8 --quant-method int8 --cpuinfer-threads 96 --threadpool-count 2 都会出现类似错误： 前者为： Loading model configuration... FP8 quantization config detected: format: e4m3 weight_block_size: [128, 128] Model config: {'num_experts': 256, 'num_experts_per_tok': 8, 'hidden_size': 7168, 'moe_intermediate_size': 2048, 'fp8_weight_block_size': [128, 128]} num_experts: 256 num_experts_per_tok: 8 hidden_size: 7168 moe_intermediate_size: 2048 CPU inference config: cpuinfer_threads: 96 threadpool_count: 2 Loading safetensors files from /home/netadmin/models/DeepSeek-V3.2 Loaded: model-00001-of-000163.safetensors (290 tensors) Loaded: model-00002-of-000163.safetensors (586 tensors) Loaded: model-00003-of-000163.safetensors (586 tensors) Loaded: model-00004-of-000163.safetensors (581 tensors) Loaded: model-00005-of-000163.safetensors (586 tensors) Loaded: model-00006-of-000163.safetensors (587 tensors) Loaded: model-00007-of-000163.safetensors (582 tensors) Loaded: model-00008-of-000163.safetensors (586 tensors) Loaded: model-00009-of-000163.safetensors (581 tensors) Loaded: model-00010-of-000163.safetensors (586 tensors) Loaded: model-00011-of-000163.safetensors (586 tensors) Loaded: model-00012-of-000163.safetensors (205 tensors) Loaded: model-00013-of-000163.safetensors (578 tensors) Loaded: model-00014-of-000163.safetensors (586 tensors) Loaded: model-00015-of-000163.safetensors (581 tensors) Loaded: model-00016-of-000163.safetensors (586 tensors) Loaded: model-00017-of-000163.safetensors (586 tensors) Loaded: model-00018-of-000163.safetensors (581 tensors) Loaded: model-00019-of-000163.safetensors (586 tensors) Loaded: model-00020-of-000163.safetensors (586 tensors) Loaded: model-00021-of-000163.safetensors (581 tensors) Loaded: model-00022-of-000163.safetensors (586 tensors) Loaded: model-00023-of-000163.safetensors (581 tensors) Loaded: model-00024-of-000163.safetensors (586 tensors) Loaded: model-00025-of-000163.safetensors (586 tensors) Loaded: model-00026-of-000163.safetensors (581 tensors) Loaded: model-00027-of-000163.safetensors (586 tensors) Loaded: model-00028-of-000163.safetensors (586 tensors) Loaded: model-00029-of-000163.safetensors (581 tensors) Loaded: model-00030-of-000163.safetensors (586 tensors) Loaded: model-00031-of-000163.safetensors (581 tensors) Loaded: model-00032-of-000163.safetensors (586 tensors) Loaded: model-00033-of-000163.safetensors (586 tensors) Loaded: model-00034-of-000163.safetensors (257 tensors) Loaded: model-00035-of-000163.safetensors (578 tensors) Loaded: model-00036-of-000163.safetensors (586 tensors) Loaded: model-00037-of-000163.safetensors (581 tensors) Loaded: model-00038-of-000163.safetensors (586 tensors) Loaded: model-00039-of-000163.safetensors (586 tensors) Loaded: model-00040-of-000163.safetensors (581 tensors) Loaded: model-00041-of-000163.safetensors (586 tensors) Loaded: model-00042-of-000163.safetensors (586 tensors) Loaded: model-00043-of-000163.safetensors (581 tensors) Loaded: model-00044-of-000163.safetensors (586 tensors) Loaded: model-00045-of-000163.safetensors (581 tensors) Loaded: model-00046-of-000163.safetensors (586 tensors) Loaded: model-00047-of-000163.safetensors (586 tensors) Loaded: model-00048-of-000163.safetensors (581 tensors) Loaded: model-00049-of-000163.safetensors (586 tensors) Loaded: model-00050-of-000163.safetensors (586 tensors) Loaded: model-00051-of-000163.safetensors (581 tensors) Loaded: model-00052-of-000163.safetensors (586 tensors) Loaded: model-00053-of-000163.safetensors (581 tensors) Loaded: model-00054-of-000163.safetensors (586 tensors) Loaded: model-00055-of-000163.safetensors (586 tensors) Loaded: model-00056-of-000163.safetensors (257 tensors) Loaded: model-00057-of-000163.safetensors (578 tensors) Loaded: model-00058-of-000163.safetensors (586 tensors) Loaded: model-00059-of-000163.safetensors (581 tensors) Loaded: model-00060-of-000163.safetensors (586 tensors) Loaded: model-00061-of-000163.safetensors (586 tensors) Loaded: model-00062-of-000163.safetensors (581 tensors) Loaded: model-00063-of-000163.safetensors (586 tensors) Loaded: model-00064-of-000163.safetensors (586 tensors) Loaded: model-00065-of-000163.safetensors (581 tensors) Loaded: model-00066-of-000163.safetensors (586 tensors) Loaded: model-00067-of-000163.safetensors (581 tensors) Loaded: model-00068-of-000163.safetensors (586 tensors) Loaded: model-00069-of-000163.safetensors (586 tensors) Loaded: model-00070-of-000163.safetensors (581 tensors) Loaded: model-00071-of-000163.safetensors (586 tensors) Loaded: model-00072-of-000163.safetensors (586 tensors) Loaded: model-00073-of-000163.safetensors (581 tensors) Loaded: model-00074-of-000163.safetensors (586 tensors) Loaded: model-00075-of-000163.safetensors (581 tensors) Loaded: model-00076-of-000163.safetensors (586 tensors) Loaded: model-00077-of-000163.safetensors (586 tensors) Loaded: model-00078-of-000163.safetensors (257 tensors) Loaded: model-00079-of-000163.safetensors (578 tensors) Loaded: model-00080-of-000163.safetensors (586 tensors) Loaded: model-00081-of-000163.safetensors (581 tensors) Loaded: model-00082-of-000163.safetensors (586 tensors) Loaded: model-00083-of-000163.safetensors (586 tensors) Loaded: model-00084-of-000163.safetensors (581 tensors) Loaded: model-00085-of-000163.safetensors (586 tensors) Loaded: model-00086-of-000163.safetensors (586 tensors) Loaded: model-00087-of-000163.safetensors (581 tensors) Loaded: model-00088-of-000163.safetensors (586 tensors) Loaded: model-00089-of-000163.safetensors (581 tensors) Loaded: model-00090-of-000163.safetensors (586 tensors) Loaded: model-00091-of-000163.safetensors (586 tensors) Loaded: model-00092-of-000163.safetensors (581 tensors) Loaded: model-00093-of-000163.safetensors (586 tensors) Loaded: model-00094-of-000163.safetensors (586 tensors) Loaded: model-00095-of-000163.safetensors (581 tensors) Loaded: model-00096-of-000163.safetensors (586 tensors) Loaded: model-00097-of-000163.safetensors (581 tensors) Loaded: model-00098-of-000163.safetensors (586 tensors) Loaded: model-00099-of-000163.safetensors (586 tensors) Loaded: model-00100-of-000163.safetensors (257 tensors) Loaded: model-00101-of-000163.safetensors (578 tensors) Loaded: model-00102-of-000163.safetensors (586 tensors) Loaded: model-00103-of-000163.safetensors (581 tensors) Loaded: model-00104-of-000163.safetensors (586 tensors) Loaded: model-00105-of-000163.safetensors (586 tensors) Loaded: model-00106-of-000163.safetensors (581 tensors) Loaded: model-00107-of-000163.safetensors (586 tensors) Loaded: model-00108-of-000163.safetensors (586 tensors) Loaded: model-00109-of-000163.safetensors (581 tensors) Loaded: model-00110-of-000163.safetensors (586 tensors) Loaded: model-00111-of-000163.safetensors (581 tensors) Loaded: model-00112-of-000163.safetensors (586 tensors) Loaded: model-00113-of-000163.safetensors (586 tensors) Loaded: model-00114-of-000163.safetensors (581 tensors) Loaded: model-00115-of-000163.safetensors (586 tensors) Loaded: model-00116-of-000163.safetensors (586 tensors) Loaded: model-00117-of-000163.safetensors (581 tensors) Loaded: model-00118-of-000163.safetensors (586 tensors) Loaded: model-00119-of-000163.safetensors (581 tensors) Loaded: model-00120-of-000163.safetensors (586 tensors) Loaded: model-00121-of-000163.safetensors (586 tensors) Loaded: model-00122-of-000163.safetensors (257 tensors) Loaded: model-00123-of-000163.safetensors (578 tensors) Loaded: model-00124-of-000163.safetensors (586 tensors) Loaded: model-00125-of-000163.safetensors (581 tensors) Loaded: model-00126-of-000163.safetensors (586 tensors) Loaded: model-00127-of-000163.safetensors (586 tensors) Loaded: model-00128-of-000163.safetensors (581 tensors) Loaded: model-00129-of-000163.safetensors (586 tensors) Loaded: model-00130-of-000163.safetensors (586 tensors) Loaded: model-00131-of-000163.safetensors (581 tensors) Loaded: model-00132-of-000163.safetensors (586 tensors) Loaded: model-00133-of-000163.safetensors (581 tensors) Loaded: model-00134-of-000163.safetensors (586 tensors) Loaded: model-00135-of-000163.safetensors (586 tensors) Loaded: model-00136-of-000163.safetensors (581 tensors) Loaded: model-00137-of-000163.safetensors (586 tensors) Loaded: model-00138-of-000163.safetensors (586 tensors) Loaded: model-00139-of-000163.safetensors (581 tensors) Loaded: model-00140-of-000163.safetensors (586 tensors) Loaded: model-00141-of-000163.safetensors (445 tensors) Loaded: model-00142-of-000163.safetensors (578 tensors) Loaded: model-00143-of-000163.safetensors (586 tensors) Loaded: model-00144-of-000163.safetensors (581 tensors) Loaded: model-00145-of-000163.safetensors (586 tensors) Loaded: model-00146-of-000163.safetensors (586 tensors) Loaded: model-00147-of-000163.safetensors (581 tensors) Loaded: model-00148-of-000163.safetensors (586 tensors) Loaded: model-00149-of-000163.safetensors (586 tensors) Loaded: model-00150-of-000163.safetensors (581 tensors) Loaded: model-00151-of-000163.safetensors (586 tensors) Loaded: model-00152-of-000163.safetensors (581 tensors) Loaded: model-00153-of-000163.safetensors (586 tensors) Loaded: model-00154-of-000163.safetensors (586 tensors) Loaded: model-00155-of-000163.safetensors (581 tensors) Loaded: model-00156-of-000163.safetensors (586 tensors) Loaded: model-00157-of-000163.safetensors (586 tensors) Loaded: model-00158-of-000163.safetensors (581 tensors) Loaded: model-00159-of-000163.safetensors (586 tensors) Loaded: model-00160-of-000163.safetensors (458 tensors) Loaded: model-00161-of-000163.safetensors (593 tensors) Loaded: model-00162-of-000163.safetensors (586 tensors) Loaded: model-00163-of-000163.safetensors (381 tensors) Total tensors loaded: 92425 Starting conversion... Input: /home/netadmin/models/DeepSeek-V3.2 Output: /home/netadmin/models/DeepSeek-V3.2-INT8 Found 59 layers with MoE experts: Layer 3: 256 experts (0-255) Layer 4: 256 experts (0-255) Layer 5: 256 experts (0-255) Layer 6: 256 experts (0-255) Layer 7: 256 experts (0-255) Layer 8: 256 experts (0-255) Layer 9: 256 experts (0-255) Layer 10: 256 experts (0-255) Layer 11: 256 experts (0-255) Layer 12: 256 experts (0-255) Layer 13: 256 experts (0-255) Layer 14: 256 experts (0-255) Layer 15: 256 experts (0-255) Layer 16: 256 experts (0-255) Layer 17: 256 experts (0-255) Layer 18: 256 experts (0-255) Layer 19: 256 experts (0-255) Layer 20: 256 experts (0-255) Layer 21: 256 experts (0-255) Layer 22: 256 experts (0-255) Layer 23: 256 experts (0-255) Layer 24: 256 experts (0-255) Layer 25: 256 experts (0-255) Layer 26: 256 experts (0-255) Layer 27: 256 experts (0-255) Layer 28: 256 experts (0-255) Layer 29: 256 experts (0-255) Layer 30: 256 experts (0-255) Layer 31: 256 experts (0-255) Layer 32: 256 experts (0-255) Layer 33: 256 experts (0-255) Layer 34: 256 experts (0-255) Layer 35: 256 experts (0-255) Layer 36: 256 experts (0-255) Layer 37: 256 experts (0-255) Layer 38: 256 experts (0-255) Layer 39: 256 experts (0-255) Layer 40: 256 experts (0-255) Layer 41: 256 experts (0-255) Layer 42: 256 experts (0-255) Layer 43: 256 experts (0-255) Layer 44: 256 experts (0-255) Layer 45: 256 experts (0-255) Layer 46: 256 experts (0-255) Layer 47: 256 experts (0-255) Layer 48: 256 experts (0-255) Layer 49: 256 experts (0-255) Layer 50: 256 experts (0-255) Layer 51: 256 experts (0-255) Layer 52: 256 experts (0-255) Layer 53: 256 experts (0-255) Layer 54: 256 experts (0-255) Layer 55: 256 experts (0-255) Layer 56: 256 experts (0-255) Layer 57: 256 experts (0-255) Layer 58: 256 experts (0-255) Layer 59: 256 experts (0-255) Layer 60: 256 experts (0-255) Layer 61: 256 experts (0-255) Processing layer 3 (1/59)... Converting layer 3 with 256 experts via online quantization... Loaded weights shapes: gate_proj: torch.Size([256, 2048, 7168]) up_proj: torch.Size([256, 2048, 7168]) down_proj: torch.Size([256, 7168, 2048]) CPUInfer[0x4394d820]: Hello WorkerPool[0x43a02ff0] 2 subpools, [numa:threads][0:48] [1:48] ===========In NumaPool============ In Numa Worker Pool at NUMA 0, 48 threads ===========In NumaPool============ In Numa Worker Pool at NUMA 1, 48 threads TP MOE layer 3, pool: 0x43a02ff0, expert num: 256, num_experts_per_tok: 8 From BF16 online quant from bf16 terminate called after throwing an instance of 'std::logic_error' terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively what(): basic_string::_M_construct null not valid terminate called recursively ✗ Quantization failed with exit code -6 后者为 python convert_cpu_weights.py --input-path /home/netadmin/models/DeepSeek-V3.2 --input-type fp8 --output /home/netadmin/models/DeepSeek-V3.2-INT8 --quant-method int8 --cpuinfer-threads 96 --threadpool-count 2 Loading model configuration... FP8 quantization config detected: format: e4m3 weight_block_size: [128, 128] Model config: {'num_experts': 256, 'num_experts_per_tok': 8, 'hidden_size': 7168, 'moe_intermediate_size': 2048, 'fp8_weight_block_size': [128, 128]} num_experts: 256 num_experts_per_tok: 8 hidden_size: 7168 moe_intermediate_size: 2048 CPU inference config: cpuinfer_threads: 96 threadpool_count: 2 Loading safetensors files from /home/netadmin/models/DeepSeek-V3.2 Loaded: model-00001-of-000163.safetensors (290 tensors) Loaded: model-00002-of-000163.safetensors (586 tensors) Loaded: model-00003-of-000163.safetensors (586 tensors) Loaded: model-00004-of-000163.safetensors (581 tensors) Loaded: model-00005-of-000163.safetensors (586 tensors) Loaded: model-00006-of-000163.safetensors (587 tensors) Loaded: model-00007-of-000163.safetensors (582 tensors) Loaded: model-00008-of-000163.safetensors (586 tensors) Loaded: model-00009-of-000163.safetensors (581 tensors) Loaded: model-00010-of-000163.safetensors (586 tensors) Loaded: model-00011-of-000163.safetensors (586 tensors) Loaded: model-00012-of-000163.safetensors (205 tensors) Loaded: model-00013-of-000163.safetensors (578 tensors) Loaded: model-00014-of-000163.safetensors (586 tensors) Loaded: model-00015-of-000163.safetensors (581 tensors) Loaded: model-00016-of-000163.safetensors (586 tensors) Loaded: model-00017-of-000163.safetensors (586 tensors) Loaded: model-00018-of-000163.safetensors (581 tensors) Loaded: model-00019-of-000163.safetensors (586 tensors) Loaded: model-00020-of-000163.safetensors (586 tensors) Loaded: model-00021-of-000163.safetensors (581 tensors) Loaded: model-00022-of-000163.safetensors (586 tensors) Loaded: model-00023-of-000163.safetensors (581 tensors) Loaded: model-00024-of-000163.safetensors (586 tensors) Loaded: model-00025-of-000163.safetensors (586 tensors) Loaded: model-00026-of-000163.safetensors (581 tensors) Loaded: model-00027-of-000163.safetensors (586 tensors) Loaded: model-00028-of-000163.safetensors (586 tensors) Loaded: model-00029-of-000163.safetensors (581 tensors) Loaded: model-00030-of-000163.safetensors (586 tensors) Loaded: model-00031-of-000163.safetensors (581 tensors) Loaded: model-00032-of-000163.safetensors (586 tensors) Loaded: model-00033-of-000163.safetensors (586 tensors) Loaded: model-00034-of-000163.safetensors (257 tensors) Loaded: model-00035-of-000163.safetensors (578 tensors) Loaded: model-00036-of-000163.safetensors (586 tensors) Loaded: model-00037-of-000163.safetensors (581 tensors) Loaded: model-00038-of-000163.safetensors (586 tensors) Loaded: model-00039-of-000163.safetensors (586 tensors) Loaded: model-00040-of-000163.safetensors (581 tensors) Loaded: model-00041-of-000163.safetensors (586 tensors) Loaded: model-00042-of-000163.safetensors (586 tensors) Loaded: model-00043-of-000163.safetensors (581 tensors) Loaded: model-00044-of-000163.safetensors (586 tensors) Loaded: model-00045-of-000163.safetensors (581 tensors) Loaded: model-00046-of-000163.safetensors (586 tensors) Loaded: model-00047-of-000163.safetensors (586 tensors) Loaded: model-00048-of-000163.safetensors (581 tensors) Loaded: model-00049-of-000163.safetensors (586 tensors) Loaded: model-00050-of-000163.safetensors (586 tensors) Loaded: model-00051-of-000163.safetensors (581 tensors) Loaded: model-00052-of-000163.safetensors (586 tensors) Loaded: model-00053-of-000163.safetensors (581 tensors) Loaded: model-00054-of-000163.safetensors (586 tensors) Loaded: model-00055-of-000163.safetensors (586 tensors) Loaded: model-00056-of-000163.safetensors (257 tensors) Loaded: model-00057-of-000163.safetensors (578 tensors) Loaded: model-00058-of-000163.safetensors (586 tensors) Loaded: model-00059-of-000163.safetensors (581 tensors) Loaded: model-00060-of-000163.safetensors (586 tensors) Loaded: model-00061-of-000163.safetensors (586 tensors) Loaded: model-00062-of-000163.safetensors (581 tensors) Loaded: model-00063-of-000163.safetensors (586 tensors) Loaded: model-00064-of-000163.safetensors (586 tensors) Loaded: model-00065-of-000163.safetensors (581 tensors) Loaded: model-00066-of-000163.safetensors (586 tensors) Loaded: model-00067-of-000163.safetensors (581 tensors) Loaded: model-00068-of-000163.safetensors (586 tensors) Loaded: model-00069-of-000163.safetensors (586 tensors) Loaded: model-00070-of-000163.safetensors (581 tensors) Loaded: model-00071-of-000163.safetensors (586 tensors) Loaded: model-00072-of-000163.safetensors (586 tensors) Loaded: model-00073-of-000163.safetensors (581 tensors) Loaded: model-00074-of-000163.safetensors (586 tensors) Loaded: model-00075-of-000163.safetensors (581 tensors) Loaded: model-00076-of-000163.safetensors (586 tensors) Loaded: model-00077-of-000163.safetensors (586 tensors) Loaded: model-00078-of-000163.safetensors (257 tensors) Loaded: model-00079-of-000163.safetensors (578 tensors) Loaded: model-00080-of-000163.safetensors (586 tensors) Loaded: model-00081-of-000163.safetensors (581 tensors) Loaded: model-00082-of-000163.safetensors (586 tensors) Loaded: model-00083-of-000163.safetensors (586 tensors) Loaded: model-00084-of-000163.safetensors (581 tensors) Loaded: model-00085-of-000163.safetensors (586 tensors) Loaded: model-00086-of-000163.safetensors (586 tensors) Loaded: model-00087-of-000163.safetensors (581 tensors) Loaded: model-00088-of-000163.safetensors (586 tensors) Loaded: model-00089-of-000163.safetensors (581 tensors) Loaded: model-00090-of-000163.safetensors (586 tensors) Loaded: model-00091-of-000163.safetensors (586 tensors) Loaded: model-00092-of-000163.safetensors (581 tensors) Loaded: model-00093-of-000163.safetensors (586 tensors) Loaded: model-00094-of-000163.safetensors (586 tensors) Loaded: model-00095-of-000163.safetensors (581 tensors) Loaded: model-00096-of-000163.safetensors (586 tensors) Loaded: model-00097-of-000163.safetensors (581 tensors) Loaded: model-00098-of-000163.safetensors (586 tensors) Loaded: model-00099-of-000163.safetensors (586 tensors) Loaded: model-00100-of-000163.safetensors (257 tensors) Loaded: model-00101-of-000163.safetensors (578 tensors) Loaded: model-00102-of-000163.safetensors (586 tensors) Loaded: model-00103-of-000163.safetensors (581 tensors) Loaded: model-00104-of-000163.safetensors (586 tensors) Loaded: model-00105-of-000163.safetensors (586 tensors) Loaded: model-00106-of-000163.safetensors (581 tensors) Loaded: model-00107-of-000163.safetensors (586 tensors) Loaded: model-00108-of-000163.safetensors (586 tensors) Loaded: model-00109-of-000163.safetensors (581 tensors) Loaded: model-00110-of-000163.safetensors (586 tensors) Loaded: model-00111-of-000163.safetensors (581 tensors) Loaded: model-00112-of-000163.safetensors (586 tensors) Loaded: model-00113-of-000163.safetensors (586 tensors) Loaded: model-00114-of-000163.safetensors (581 tensors) Loaded: model-00115-of-000163.safetensors (586 tensors) Loaded: model-00116-of-000163.safetensors (586 tensors) Loaded: model-00117-of-000163.safetensors (581 tensors) Loaded: model-00118-of-000163.safetensors (586 tensors) Loaded: model-00119-of-000163.safetensors (581 tensors) Loaded: model-00120-of-000163.safetensors (586 tensors) Loaded: model-00121-of-000163.safetensors (586 tensors) Loaded: model-00122-of-000163.safetensors (257 tensors) Loaded: model-00123-of-000163.safetensors (578 tensors) Loaded: model-00124-of-000163.safetensors (586 tensors) Loaded: model-00125-of-000163.safetensors (581 tensors) Loaded: model-00126-of-000163.safetensors (586 tensors) Loaded: model-00127-of-000163.safetensors (586 tensors) Loaded: model-00128-of-000163.safetensors (581 tensors) Loaded: model-00129-of-000163.safetensors (586 tensors) Loaded: model-00130-of-000163.safetensors (586 tensors) Loaded: model-00131-of-000163.safetensors (581 tensors) Loaded: model-00132-of-000163.safetensors (586 tensors) Loaded: model-00133-of-000163.safetensors (581 tensors) Loaded: model-00134-of-000163.safetensors (586 tensors) Loaded: model-00135-of-000163.safetensors (586 tensors) Loaded: model-00136-of-000163.safetensors (581 tensors) Loaded: model-00137-of-000163.safetensors (586 tensors) Loaded: model-00138-of-000163.safetensors (586 tensors) Loaded: model-00139-of-000163.safetensors (581 tensors) Loaded: model-00140-of-000163.safetensors (586 tensors) Loaded: model-00141-of-000163.safetensors (445 tensors) Loaded: model-00142-of-000163.safetensors (578 tensors) Loaded: model-00143-of-000163.safetensors (586 tensors) Loaded: model-00144-of-000163.safetensors (581 tensors) Loaded: model-00145-of-000163.safetensors (586 tensors) Loaded: model-00146-of-000163.safetensors (586 tensors) Loaded: model-00147-of-000163.safetensors (581 tensors) Loaded: model-00148-of-000163.safetensors (586 tensors) Loaded: model-00149-of-000163.safetensors (586 tensors) Loaded: model-00150-of-000163.safetensors (581 tensors) Loaded: model-00151-of-000163.safetensors (586 tensors) Loaded: model-00152-of-000163.safetensors (581 tensors) Loaded: model-00153-of-000163.safetensors (586 tensors) Loaded: model-00154-of-000163.safetensors (586 tensors) Loaded: model-00155-of-000163.safetensors (581 tensors) Loaded: model-00156-of-000163.safetensors (586 tensors) Loaded: model-00157-of-000163.safetensors (586 tensors) Loaded: model-00158-of-000163.safetensors (581 tensors) Loaded: model-00159-of-000163.safetensors (586 tensors) Loaded: model-00160-of-000163.safetensors (458 tensors) Loaded: model-00161-of-000163.safetensors (593 tensors) Loaded: model-00162-of-000163.safetensors (586 tensors) Loaded: model-00163-of-000163.safetensors (381 tensors) Total tensors loaded: 92425 Starting conversion... Input: /home/netadmin/models/DeepSeek-V3.2 Output: /home/netadmin/models/DeepSeek-V3.2-INT8 Found 59 layers with MoE experts: Layer 3: 256 experts (0-255) Layer 4: 256 experts (0-255) Layer 5: 256 experts (0-255) Layer 6: 256 experts (0-255) Layer 7: 256 experts (0-255) Layer 8: 256 experts (0-255) Layer 9: 256 experts (0-255) Layer 10: 256 experts (0-255) Layer 11: 256 experts (0-255) Layer 12: 256 experts (0-255) Layer 13: 256 experts (0-255) Layer 14: 256 experts (0-255) Layer 15: 256 experts (0-255) Layer 16: 256 experts (0-255) Layer 17: 256 experts (0-255) Layer 18: 256 experts (0-255) Layer 19: 256 experts (0-255) Layer 20: 256 experts (0-255) Layer 21: 256 experts (0-255) Layer 22: 256 experts (0-255) Layer 23: 256 experts (0-255) Layer 24: 256 experts (0-255) Layer 25: 256 experts (0-255) Layer 26: 256 experts (0-255) Layer 27: 256 experts (0-255) Layer 28: 256 experts (0-255) Layer 29: 256 experts (0-255) Layer 30: 256 experts (0-255) Layer 31: 256 experts (0-255) Layer 32: 256 experts (0-255) Layer 33: 256 experts (0-255) Layer 34: 256 experts (0-255) Layer 35: 256 experts (0-255) Layer 36: 256 experts (0-255) Layer 37: 256 experts (0-255) Layer 38: 256 experts (0-255) Layer 39: 256 experts (0-255) Layer 40: 256 experts (0-255) Layer 41: 256 experts (0-255) Layer 42: 256 experts (0-255) Layer 43: 256 experts (0-255) Layer 44: 256 experts (0-255) Layer 45: 256 experts (0-255) Layer 46: 256 experts (0-255) Layer 47: 256 experts (0-255) Layer 48: 256 experts (0-255) Layer 49: 256 experts (0-255) Layer 50: 256 experts (0-255) Layer 51: 256 experts (0-255) Layer 52: 256 experts (0-255) Layer 53: 256 experts (0-255) Layer 54: 256 experts (0-255) Layer 55: 256 experts (0-255) Layer 56: 256 experts (0-255) Layer 57: 256 experts (0-255) Layer 58: 256 experts (0-255) Layer 59: 256 experts (0-255) Layer 60: 256 experts (0-255) Layer 61: 256 experts (0-255) Processing layer 3 (1/59)... Converting layer 3 with 256 experts via online quantization... Loaded weights shapes: gate_proj: torch.Size([256, 2048, 7168]) up_proj: torch.Size([256, 2048, 7168]) down_proj: torch.Size([256, 7168, 2048]) CPUInfer[0x154dfd80]: Hello WorkerPool[0x154dfd10] 2 subpools, [numa:threads][0:48] [1:48] ===========In NumaPool============ In Numa Worker Pool at NUMA 0, 48 threads ===========In NumaPool============ In Numa Worker Pool at NUMA 1, 48 threads TP MOE layer 3, pool: 0x154dfd10, expert num: 256, num_experts_per_tok: 8 From BF16 online quant from bf16 terminate called after throwing an instance of 'std::logic_error' terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively terminate called recursively 已中止 (核心已转储)
bsakaguchi commented 2026-10-29 14:48:48 +00:00
Author
Owner
Copy link

@Jiqiren87 commented on GitHub (Jan 18, 2026):

问题已经解决。pip安装不可行，git后安装问题解决。

<!-- gh-comment-id:3765212844 --> @Jiqiren87 commented on GitHub (Jan 18, 2026): 问题已经解决。pip安装不可行，git后安装问题解决。
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#1122
No description provided.