[GH-ISSUE #471] 权重文件加载很慢是什么原因？ #331

New issue

Closed

opened 2026-10-29 14:37:59 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:37:59 +00:00

Owner

Originally created by @burugo on GitHub (Feb 18, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/471

CPU型号是Intel(R) Xeon(R) Platinum 8179M CPU @ 2.40GHz
有一张3090显卡，显存24G
内存DDR 4 ，2666 MT/s，8根64G=512G
运行Q4满血版的gguf
加载权重文件非常慢，几十分钟了还没加载完
log显示大概一分多钟才能出现新的blk数字：

loading blk.48.attn_q_a_norm.weight to cuda:0
loading blk.48.attn_kv_a_norm.weight to cuda:0
loading blk.48.attn_kv_b.weight to cuda:0
loading blk.48.attn_norm.weight to cuda:0
loading blk.48.ffn_norm.weight to cuda:0
loading blk.49.attn_q_a_norm.weight to cuda:0
loading blk.49.attn_kv_a_norm.weight to cuda:0
loading blk.49.attn_kv_b.weight to cuda:0
loading blk.49.attn_norm.weight to cuda:0
loading blk.49.ffn_norm.weight to cuda:0
loading blk.50.attn_q_a_norm.weight to cuda:0
loading blk.50.attn_kv_a_norm.weight to cuda:0
loading blk.50.attn_kv_b.weight to cuda:0

加载过程中进程显示内存占用70%

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
2577178 being     20   0  419.5g 363.2g 358.8g R  2257  72.1 417:19.23 python

Originally created by @burugo on GitHub (Feb 18, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/471 CPU型号是Intel(R) Xeon(R) Platinum 8179M CPU @ 2.40GHz 有一张3090显卡，显存24G 内存DDR 4 ，2666 MT/s，8根64G=512G 运行Q4满血版的gguf 加载权重文件非常慢，几十分钟了还没加载完 log显示大概一分多钟才能出现新的blk数字： ``` loading blk.48.attn_q_a_norm.weight to cuda:0 loading blk.48.attn_kv_a_norm.weight to cuda:0 loading blk.48.attn_kv_b.weight to cuda:0 loading blk.48.attn_norm.weight to cuda:0 loading blk.48.ffn_norm.weight to cuda:0 loading blk.49.attn_q_a_norm.weight to cuda:0 loading blk.49.attn_kv_a_norm.weight to cuda:0 loading blk.49.attn_kv_b.weight to cuda:0 loading blk.49.attn_norm.weight to cuda:0 loading blk.49.ffn_norm.weight to cuda:0 loading blk.50.attn_q_a_norm.weight to cuda:0 loading blk.50.attn_kv_a_norm.weight to cuda:0 loading blk.50.attn_kv_b.weight to cuda:0 ``` 加载过程中进程显示内存占用70% ``` PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2577178 being 20 0 419.5g 363.2g 358.8g R 2257 72.1 417:19.23 python ```

bsakaguchi closed this issue

2026-10-29 14:38:00 +00:00

bsakaguchi commented

2026-10-29 14:38:00 +00:00

Author

Owner

@burugo commented on GitHub (Feb 18, 2025):

大概知道原因了，文件放在机械硬盘上

@burugo commented on GitHub (Feb 18, 2025): 大概知道原因了，文件放在机械硬盘上

bsakaguchi commented

2026-10-29 14:38:00 +00:00

Author