mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1842] MiniMax2.1输出很慢很慢很慢，结果也是乱码，麻烦指点一下，谢谢先 #1150
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#1150
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @zcc2xj on GitHub (Feb 11, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1842
Reminder
System Info
运行配置情况：
KTransformers 环境诊断
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Check ┃ Status ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Python 版本 │ 正常 │ 3.12.12 │
│ CUDA 可用性 │ 正常 │ 13.0 │
│ GPU 检测 │ 正常 │ 发现 2 个 GPU: NVIDIA GeForce RTX 4090, NVIDIA GeForce RTX 4090 │
│ CPU │ 正常 │ AMD EPYC 9654 96-Core Processor (192 核心 / 384 线程) │
│ CPU 指令集 │ 正常 │ AVX512BF16, AVX512F, AVX512BW, AVX512VL, AVX2, AVX, FMA, SSE4.2 (+9 more) │
│ NUMA 拓扑 │ 正常 │ 8 个节点 │
│ kt-kernel │ 正常 │ v0.5.1 (AVX512_BF16) │
│ 系统内存 │ 正常 │ 848.3GB 可用 / 1086.2GB 总计 │
│ 磁盘空间 │ 正常 │ /media/firefly/Backup1/Python/Ktransformers/models 有 13512.6GB 可用空间 │
│ SGLang Source │ 正常 │ Source (GitHub: kvcache-ai/sglang, branch: main) │
│ SGLang kt-kernel │ 正常 │ SGLang kt-kernel 支持已验证 │
│ Environment Managers │ 正常 │ conda 25.11.1, venv builtin │
└──────────────────────┴──────────┴───────────────────────────────────────────────────────────────────────────┘
输出很慢：
[2026-02-11 10:49:01 TP0] Prefill batch, #new-seq: 1, #new-token: 8, #cached-token: 31, token usage: 0.00, #running-req: 0, #queue-req: 0, input throughput (token/s): 0.03, cuda graph: False
[2026-02-11 10:49:28 TP0] Decode batch, #running-req: 1, #token: 48, token usage: 0.00, cuda graph: True, gen throughput (token/s): 0.20, #queue-req: 0
[2026-02-11 10:51:52 TP0] Decode batch, #running-req: 1, #token: 88, token usage: 0.00, cuda graph: True, gen throughput (token/s): 0.28, #queue-req: 0
[2026-02-11 10:54:13 TP0] Decode batch, #running-req: 1, #token: 128, token usage: 0.00, cuda graph: True, gen throughput (token/s): 0.28, #queue-req: 0
[2026-02-11 10:56:37 TP0] Decode batch, #running-req: 1, #token: 168, token usage: 0.00, cuda graph: True, gen throughput (token/s): 0.28, #queue-req: 0
[2026-02-11 10:58:59 TP0] Decode batch, #running-req: 1, #token: 208, token usage: 0.00, cuda graph: True, gen throughput (token/s): 0.28, #queue-req: 0
[2026-02-11 11:01:21 TP0] Decode batch, #running-req: 1, #token: 248, token usage: 0.00, cuda graph: True, gen throughput (token/s): 0.28, #queue-req: 0
输出还是乱码：
Reproduction
@ouqingliang commented on GitHub (Feb 12, 2026):
可能是 NUMA 数量太多了，MiniMax-M2.1 纯 tp 只能切 4 份。可以尝试将
--kt-threadpool-count设成 4 试试？@JinghuaGao commented on GitHub (Feb 18, 2026):
量化的过于严重了？
@ErvinXie commented on GitHub (Feb 28, 2026):
0.2 tok/s 的 decode 速度明显异常，说明可能存在配置问题。可参考官网 benchmark 作为对比：https://ktransformers.net/zh/benchmarks#MiniMax-M2.1-FP8-TP4
请提供以下信息以便排查：
numactl --hardware的输出另外，EPYC 9654 通常有多个 NUMA node，建议确认
--kt-threadpool-count设置与 NUMA node 数量匹配。This reply was co-authored by an AI agent and a human.
@merries711 commented on GitHub (Mar 16, 2026):
我的服务器也是这样，命令用的是官网benchmark页面上的。
System Info
CPU: INTEL(R) XEON(R) GOLD 6530 x2
GPU: NVIDIA RTX PRO 4500 Blackwell 32GB [ GB203GL ] x2
RAM: 1TB DDR5
KTransformers CLI v0.1.0
Python 3.11.14
平台 Linux 6.12.63+deb13-amd64
CUDA 13.1
Packages:
kt-kernel 0.5.2.post1
sglang-kt 0.5.2.post2 (sglang-kt)
installed from source .
1、numactl --hardware 的输出：
2、完整的启动命令（改成适配的 --kt-cpuinfer 60 --kt-threadpool-count 4，也是同样的现象）：
3、服务启动时的完整日志
@ErvinXie
@ErvinXie commented on GitHub (Mar 17, 2026):
@merries711 Thanks for the detailed info.
Looking at your logs, there are several issues:
1. NUMA configuration — disable SNC in BIOS
Your Xeon 6530 x2 shows 4 NUMA nodes, which means your BIOS has SNC (Sub-NUMA Clustering) enabled — each physical CPU socket is split into 2 NUMA nodes, resulting in only 16 cores per node.
Recommended: Go into BIOS and disable SNC (set to "SNC disabled" or "Whole CPU" mode). This will give you 2 NUMA nodes (one per socket) with 32 cores each, which is much better for KTransformers' thread pool layout.
After disabling SNC, use:
Also note that NUMA node 2 currently has almost no free memory (1114 MB vs 258 GB total) — this alone can cause severe performance degradation. Disabling SNC and ensuring memory is evenly distributed across sockets will help.
2. GPU memory pressure
With 2× 32GB GPUs,
--kt-num-gpu-experts 31is very aggressive for MiniMax-M2.1. This likely leaves very little room for KV cache, explaining the 0.1 tok/s decode speed and warmup crash.Try reducing significantly:
See also #1885 for additional details about your Blackwell GPU compatibility.
This reply was co-authored by an AI agent and a human.