kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #616] 使用最新版本0.2.1，需要flash-attention的什么版本呢？是否还支持internlm2_5-7b-chat-1m模型？ #456

New issue
Open
opened 2026-10-29 14:39:54 +00:00 by bsakaguchi · 7 comments
bsakaguchi commented 2026-10-29 14:39:54 +00:00
Owner
Copy link

Originally created by @4thGardenOfQMH on GitHub (Feb 23, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/616

Originally assigned to: @Azure-Tang, @ovowei, @Atream on GitHub.

我试图使用命令 python ./ktransformers/local_chat.py --model_path /host/internlm2_5-7b-chat-1m --gguf_path /host/internlm2_5-7b-chat-1m-gguf --cpu_infer 33 --max_new_tokens 1000 --mode "/***/ktransformers-0.2.1/ktransformers/optimize/optimize_rules/Internlm2_5-7b-Chat-1m.yaml"启动internlm2_5-7b-chat-1m模型，使用internlm2_5-7b-chat-1m-q4_k_m.gguf，下载的是上海人工智能实验室提供的量化后版本，但是遇到了如下报错，请问是模型有问题，还是不支持呢？如果还需要什么信息，请告诉我，多谢！

`
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
please input the path of your rule file(yaml file containing optimize rules):/host/ktransformers-0.2.1/ktransformers/optimize/optimize_rules/Internlm2_5-7b-Chat-1m.yaml

Injecting model as default
Injecting model.tok_embeddings as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.attention as default
Injecting model.layers.0.attention.wqkv as default
Injecting model.layers.0.attention.wo as default
Injecting model.layers.0.attention.rotary_emb as default
Injecting model.layers.0.feed_forward as default
Injecting model.layers.0.feed_forward.w1 as default
Injecting model.layers.0.feed_forward.w3 as default
Injecting model.layers.0.feed_forward.w2 as default
Injecting model.layers.0.feed_forward.act_fn as default
Injecting model.layers.0.attention_norm as default
Injecting model.layers.0.ffn_norm as default
Injecting model.layers.1 as default
Injecting model.layers.1.attention as default
Injecting model.layers.1.attention.wqkv as default
Injecting model.layers.1.attention.wo as default
Injecting model.layers.1.attention.rotary_emb as default
Injecting model.layers.1.feed_forward as default
Injecting model.layers.1.feed_forward.w1 as default
Injecting model.layers.1.feed_forward.w3 as default
Injecting model.layers.1.feed_forward.w2 as default
Injecting model.layers.1.feed_forward.act_fn as default
Injecting model.layers.1.attention_norm as default
Injecting model.layers.1.ffn_norm as default
Injecting model.layers.2 as default
Injecting model.layers.2.attention as default
Injecting model.layers.2.attention.wqkv as default
Injecting model.layers.2.attention.wo as default
Injecting model.layers.2.attention.rotary_emb as default
Injecting model.layers.2.feed_forward as default
Injecting model.layers.2.feed_forward.w1 as default
Injecting model.layers.2.feed_forward.w3 as default
Injecting model.layers.2.feed_forward.w2 as default
Injecting model.layers.2.feed_forward.act_fn as default
Injecting model.layers.2.attention_norm as default
Injecting model.layers.2.ffn_norm as default
Injecting model.layers.3 as default
Injecting model.layers.3.attention as default
Injecting model.layers.3.attention.wqkv as default
Injecting model.layers.3.attention.wo as default
Injecting model.layers.3.attention.rotary_emb as default
Injecting model.layers.3.feed_forward as default
Injecting model.layers.3.feed_forward.w1 as default
Injecting model.layers.3.feed_forward.w3 as default
Injecting model.layers.3.feed_forward.w2 as default
Injecting model.layers.3.feed_forward.act_fn as default
Injecting model.layers.3.attention_norm as default
Injecting model.layers.3.ffn_norm as default
Injecting model.layers.4 as default
Injecting model.layers.4.attention as default
Injecting model.layers.4.attention.wqkv as default
Injecting model.layers.4.attention.wo as default
Injecting model.layers.4.attention.rotary_emb as default
Injecting model.layers.4.feed_forward as default
Injecting model.layers.4.feed_forward.w1 as default
Injecting model.layers.4.feed_forward.w3 as default
Injecting model.layers.4.feed_forward.w2 as default
Injecting model.layers.4.feed_forward.act_fn as default
Injecting model.layers.4.attention_norm as default
Injecting model.layers.4.ffn_norm as default
Injecting model.layers.5 as default
Injecting model.layers.5.attention as default
Injecting model.layers.5.attention.wqkv as default
Injecting model.layers.5.attention.wo as default
Injecting model.layers.5.attention.rotary_emb as default
Injecting model.layers.5.feed_forward as default
Injecting model.layers.5.feed_forward.w1 as default
Injecting model.layers.5.feed_forward.w3 as default
Injecting model.layers.5.feed_forward.w2 as default
Injecting model.layers.5.feed_forward.act_fn as default
Injecting model.layers.5.attention_norm as default
Injecting model.layers.5.ffn_norm as default
Injecting model.layers.6 as default
Injecting model.layers.6.attention as default
Injecting model.layers.6.attention.wqkv as default
Injecting model.layers.6.attention.wo as default
Injecting model.layers.6.attention.rotary_emb as default
Injecting model.layers.6.feed_forward as default
Injecting model.layers.6.feed_forward.w1 as default
Injecting model.layers.6.feed_forward.w3 as default
Injecting model.layers.6.feed_forward.w2 as default
Injecting model.layers.6.feed_forward.act_fn as default
Injecting model.layers.6.attention_norm as default
Injecting model.layers.6.ffn_norm as default
Injecting model.layers.7 as default
Injecting model.layers.7.attention as default
Injecting model.layers.7.attention.wqkv as default
Injecting model.layers.7.attention.wo as default
Injecting model.layers.7.attention.rotary_emb as default
Injecting model.layers.7.feed_forward as default
Injecting model.layers.7.feed_forward.w1 as default
Injecting model.layers.7.feed_forward.w3 as default
Injecting model.layers.7.feed_forward.w2 as default
Injecting model.layers.7.feed_forward.act_fn as default
Injecting model.layers.7.attention_norm as default
Injecting model.layers.7.ffn_norm as default
Injecting model.layers.8 as default
Injecting model.layers.8.attention as default
Injecting model.layers.8.attention.wqkv as default
Injecting model.layers.8.attention.wo as default
Injecting model.layers.8.attention.rotary_emb as default
Injecting model.layers.8.feed_forward as default
Injecting model.layers.8.feed_forward.w1 as default
Injecting model.layers.8.feed_forward.w3 as default
Injecting model.layers.8.feed_forward.w2 as default
Injecting model.layers.8.feed_forward.act_fn as default
Injecting model.layers.8.attention_norm as default
Injecting model.layers.8.ffn_norm as default
Injecting model.layers.9 as default
Injecting model.layers.9.attention as default
Injecting model.layers.9.attention.wqkv as default
Injecting model.layers.9.attention.wo as default
Injecting model.layers.9.attention.rotary_emb as default
Injecting model.layers.9.feed_forward as default
Injecting model.layers.9.feed_forward.w1 as default
Injecting model.layers.9.feed_forward.w3 as default
Injecting model.layers.9.feed_forward.w2 as default
Injecting model.layers.9.feed_forward.act_fn as default
Injecting model.layers.9.attention_norm as default
Injecting model.layers.9.ffn_norm as default
Injecting model.layers.10 as default
Injecting model.layers.10.attention as default
Injecting model.layers.10.attention.wqkv as default
Injecting model.layers.10.attention.wo as default
Injecting model.layers.10.attention.rotary_emb as default
Injecting model.layers.10.feed_forward as default
Injecting model.layers.10.feed_forward.w1 as default
Injecting model.layers.10.feed_forward.w3 as default
Injecting model.layers.10.feed_forward.w2 as default
Injecting model.layers.10.feed_forward.act_fn as default
Injecting model.layers.10.attention_norm as default
Injecting model.layers.10.ffn_norm as default
Injecting model.layers.11 as default
Injecting model.layers.11.attention as default
Injecting model.layers.11.attention.wqkv as default
Injecting model.layers.11.attention.wo as default
Injecting model.layers.11.attention.rotary_emb as default
Injecting model.layers.11.feed_forward as default
Injecting model.layers.11.feed_forward.w1 as default
Injecting model.layers.11.feed_forward.w3 as default
Injecting model.layers.11.feed_forward.w2 as default
Injecting model.layers.11.feed_forward.act_fn as default
Injecting model.layers.11.attention_norm as default
Injecting model.layers.11.ffn_norm as default
Injecting model.layers.12 as default
Injecting model.layers.12.attention as default
Injecting model.layers.12.attention.wqkv as default
Injecting model.layers.12.attention.wo as default
Injecting model.layers.12.attention.rotary_emb as default
Injecting model.layers.12.feed_forward as default
Injecting model.layers.12.feed_forward.w1 as default
Injecting model.layers.12.feed_forward.w3 as default
Injecting model.layers.12.feed_forward.w2 as default
Injecting model.layers.12.feed_forward.act_fn as default
Injecting model.layers.12.attention_norm as default
Injecting model.layers.12.ffn_norm as default
Injecting model.layers.13 as default
Injecting model.layers.13.attention as default
Injecting model.layers.13.attention.wqkv as default
Injecting model.layers.13.attention.wo as default
Injecting model.layers.13.attention.rotary_emb as default
Injecting model.layers.13.feed_forward as default
Injecting model.layers.13.feed_forward.w1 as default
Injecting model.layers.13.feed_forward.w3 as default
Injecting model.layers.13.feed_forward.w2 as default
Injecting model.layers.13.feed_forward.act_fn as default
Injecting model.layers.13.attention_norm as default
Injecting model.layers.13.ffn_norm as default
Injecting model.layers.14 as default
Injecting model.layers.14.attention as default
Injecting model.layers.14.attention.wqkv as default
Injecting model.layers.14.attention.wo as default
Injecting model.layers.14.attention.rotary_emb as default
Injecting model.layers.14.feed_forward as default
Injecting model.layers.14.feed_forward.w1 as default
Injecting model.layers.14.feed_forward.w3 as default
Injecting model.layers.14.feed_forward.w2 as default
Injecting model.layers.14.feed_forward.act_fn as default
Injecting model.layers.14.attention_norm as default
Injecting model.layers.14.ffn_norm as default
Injecting model.layers.15 as default
Injecting model.layers.15.attention as default
Injecting model.layers.15.attention.wqkv as default
Injecting model.layers.15.attention.wo as default
Injecting model.layers.15.attention.rotary_emb as default
Injecting model.layers.15.feed_forward as default
Injecting model.layers.15.feed_forward.w1 as default
Injecting model.layers.15.feed_forward.w3 as default
Injecting model.layers.15.feed_forward.w2 as default
Injecting model.layers.15.feed_forward.act_fn as default
Injecting model.layers.15.attention_norm as default
Injecting model.layers.15.ffn_norm as default
Injecting model.layers.16 as default
Injecting model.layers.16.attention as default
Injecting model.layers.16.attention.wqkv as default
Injecting model.layers.16.attention.wo as default
Injecting model.layers.16.attention.rotary_emb as default
Injecting model.layers.16.feed_forward as default
Injecting model.layers.16.feed_forward.w1 as default
Injecting model.layers.16.feed_forward.w3 as default
Injecting model.layers.16.feed_forward.w2 as default
Injecting model.layers.16.feed_forward.act_fn as default
Injecting model.layers.16.attention_norm as default
Injecting model.layers.16.ffn_norm as default
Injecting model.layers.17 as default
Injecting model.layers.17.attention as default
Injecting model.layers.17.attention.wqkv as default
Injecting model.layers.17.attention.wo as default
Injecting model.layers.17.attention.rotary_emb as default
Injecting model.layers.17.feed_forward as default
Injecting model.layers.17.feed_forward.w1 as default
Injecting model.layers.17.feed_forward.w3 as default
Injecting model.layers.17.feed_forward.w2 as default
Injecting model.layers.17.feed_forward.act_fn as default
Injecting model.layers.17.attention_norm as default
Injecting model.layers.17.ffn_norm as default
Injecting model.layers.18 as default
Injecting model.layers.18.attention as default
Injecting model.layers.18.attention.wqkv as default
Injecting model.layers.18.attention.wo as default
Injecting model.layers.18.attention.rotary_emb as default
Injecting model.layers.18.feed_forward as default
Injecting model.layers.18.feed_forward.w1 as default
Injecting model.layers.18.feed_forward.w3 as default
Injecting model.layers.18.feed_forward.w2 as default
Injecting model.layers.18.feed_forward.act_fn as default
Injecting model.layers.18.attention_norm as default
Injecting model.layers.18.ffn_norm as default
Injecting model.layers.19 as default
Injecting model.layers.19.attention as default
Injecting model.layers.19.attention.wqkv as default
Injecting model.layers.19.attention.wo as default
Injecting model.layers.19.attention.rotary_emb as default
Injecting model.layers.19.feed_forward as default
Injecting model.layers.19.feed_forward.w1 as default
Injecting model.layers.19.feed_forward.w3 as default
Injecting model.layers.19.feed_forward.w2 as default
Injecting model.layers.19.feed_forward.act_fn as default
Injecting model.layers.19.attention_norm as default
Injecting model.layers.19.ffn_norm as default
Injecting model.layers.20 as default
Injecting model.layers.20.attention as default
Injecting model.layers.20.attention.wqkv as default
Injecting model.layers.20.attention.wo as default
Injecting model.layers.20.attention.rotary_emb as default
Injecting model.layers.20.feed_forward as default
Injecting model.layers.20.feed_forward.w1 as default
Injecting model.layers.20.feed_forward.w3 as default
Injecting model.layers.20.feed_forward.w2 as default
Injecting model.layers.20.feed_forward.act_fn as default
Injecting model.layers.20.attention_norm as default
Injecting model.layers.20.ffn_norm as default
Injecting model.layers.21 as default
Injecting model.layers.21.attention as default
Injecting model.layers.21.attention.wqkv as default
Injecting model.layers.21.attention.wo as default
Injecting model.layers.21.attention.rotary_emb as default
Injecting model.layers.21.feed_forward as default
Injecting model.layers.21.feed_forward.w1 as default
Injecting model.layers.21.feed_forward.w3 as default
Injecting model.layers.21.feed_forward.w2 as default
Injecting model.layers.21.feed_forward.act_fn as default
Injecting model.layers.21.attention_norm as default
Injecting model.layers.21.ffn_norm as default
Injecting model.layers.22 as default
Injecting model.layers.22.attention as default
Injecting model.layers.22.attention.wqkv as default
Injecting model.layers.22.attention.wo as default
Injecting model.layers.22.attention.rotary_emb as default
Injecting model.layers.22.feed_forward as default
Injecting model.layers.22.feed_forward.w1 as default
Injecting model.layers.22.feed_forward.w3 as default
Injecting model.layers.22.feed_forward.w2 as default
Injecting model.layers.22.feed_forward.act_fn as default
Injecting model.layers.22.attention_norm as default
Injecting model.layers.22.ffn_norm as default
Injecting model.layers.23 as default
Injecting model.layers.23.attention as default
Injecting model.layers.23.attention.wqkv as default
Injecting model.layers.23.attention.wo as default
Injecting model.layers.23.attention.rotary_emb as default
Injecting model.layers.23.feed_forward as default
Injecting model.layers.23.feed_forward.w1 as default
Injecting model.layers.23.feed_forward.w3 as default
Injecting model.layers.23.feed_forward.w2 as default
Injecting model.layers.23.feed_forward.act_fn as default
Injecting model.layers.23.attention_norm as default
Injecting model.layers.23.ffn_norm as default
Injecting model.layers.24 as default
Injecting model.layers.24.attention as default
Injecting model.layers.24.attention.wqkv as default
Injecting model.layers.24.attention.wo as default
Injecting model.layers.24.attention.rotary_emb as default
Injecting model.layers.24.feed_forward as default
Injecting model.layers.24.feed_forward.w1 as default
Injecting model.layers.24.feed_forward.w3 as default
Injecting model.layers.24.feed_forward.w2 as default
Injecting model.layers.24.feed_forward.act_fn as default
Injecting model.layers.24.attention_norm as default
Injecting model.layers.24.ffn_norm as default
Injecting model.layers.25 as default
Injecting model.layers.25.attention as default
Injecting model.layers.25.attention.wqkv as default
Injecting model.layers.25.attention.wo as default
Injecting model.layers.25.attention.rotary_emb as default
Injecting model.layers.25.feed_forward as default
Injecting model.layers.25.feed_forward.w1 as default
Injecting model.layers.25.feed_forward.w3 as default
Injecting model.layers.25.feed_forward.w2 as default
Injecting model.layers.25.feed_forward.act_fn as default
Injecting model.layers.25.attention_norm as default
Injecting model.layers.25.ffn_norm as default
Injecting model.layers.26 as default
Injecting model.layers.26.attention as default
Injecting model.layers.26.attention.wqkv as default
Injecting model.layers.26.attention.wo as default
Injecting model.layers.26.attention.rotary_emb as default
Injecting model.layers.26.feed_forward as default
Injecting model.layers.26.feed_forward.w1 as default
Injecting model.layers.26.feed_forward.w3 as default
Injecting model.layers.26.feed_forward.w2 as default
Injecting model.layers.26.feed_forward.act_fn as default
Injecting model.layers.26.attention_norm as default
Injecting model.layers.26.ffn_norm as default
Injecting model.layers.27 as default
Injecting model.layers.27.attention as default
Injecting model.layers.27.attention.wqkv as default
Injecting model.layers.27.attention.wo as default
Injecting model.layers.27.attention.rotary_emb as default
Injecting model.layers.27.feed_forward as default
Injecting model.layers.27.feed_forward.w1 as default
Injecting model.layers.27.feed_forward.w3 as default
Injecting model.layers.27.feed_forward.w2 as default
Injecting model.layers.27.feed_forward.act_fn as default
Injecting model.layers.27.attention_norm as default
Injecting model.layers.27.ffn_norm as default
Injecting model.layers.28 as default
Injecting model.layers.28.attention as default
Injecting model.layers.28.attention.wqkv as default
Injecting model.layers.28.attention.wo as default
Injecting model.layers.28.attention.rotary_emb as default
Injecting model.layers.28.feed_forward as default
Injecting model.layers.28.feed_forward.w1 as default
Injecting model.layers.28.feed_forward.w3 as default
Injecting model.layers.28.feed_forward.w2 as default
Injecting model.layers.28.feed_forward.act_fn as default
Injecting model.layers.28.attention_norm as default
Injecting model.layers.28.ffn_norm as default
Injecting model.layers.29 as default
Injecting model.layers.29.attention as default
Injecting model.layers.29.attention.wqkv as default
Injecting model.layers.29.attention.wo as default
Injecting model.layers.29.attention.rotary_emb as default
Injecting model.layers.29.feed_forward as default
Injecting model.layers.29.feed_forward.w1 as default
Injecting model.layers.29.feed_forward.w3 as default
Injecting model.layers.29.feed_forward.w2 as default
Injecting model.layers.29.feed_forward.act_fn as default
Injecting model.layers.29.attention_norm as default
Injecting model.layers.29.ffn_norm as default
Injecting model.layers.30 as default
Injecting model.layers.30.attention as default
Injecting model.layers.30.attention.wqkv as default
Injecting model.layers.30.attention.wo as default
Injecting model.layers.30.attention.rotary_emb as default
Injecting model.layers.30.feed_forward as default
Injecting model.layers.30.feed_forward.w1 as default
Injecting model.layers.30.feed_forward.w3 as default
Injecting model.layers.30.feed_forward.w2 as default
Injecting model.layers.30.feed_forward.act_fn as default
Injecting model.layers.30.attention_norm as default
Injecting model.layers.30.ffn_norm as default
Injecting model.layers.31 as default
Injecting model.layers.31.attention as default
Injecting model.layers.31.attention.wqkv as default
Injecting model.layers.31.attention.wo as default
Injecting model.layers.31.attention.rotary_emb as default
Injecting model.layers.31.feed_forward as default
Injecting model.layers.31.feed_forward.w1 as default
Injecting model.layers.31.feed_forward.w3 as default
Injecting model.layers.31.feed_forward.w2 as default
Injecting model.layers.31.feed_forward.act_fn as default
Injecting model.layers.31.attention_norm as default
Injecting model.layers.31.ffn_norm as default
Injecting model.norm as default
Injecting output as default
Traceback (most recent call last):
File "/host/ktransformers-0.2.1/./ktransformers/local_chat.py", line 179, in
fire.Fire(local_chat)
File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/host/ktransformers-0.2.1/./ktransformers/local_chat.py", line 110, in local_chat
optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config)
File "/host/ktransformers-0.2.1/./ktransformers/optimize/optimize.py", line 129, in optimize_and_load_gguf
load_weights(module, gguf_loader)
File "/host/ktransformers-0.2.1/./ktransformers/util/utils.py", line 85, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/host/ktransformers-0.2.1/./ktransformers/util/utils.py", line 85, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/host/ktransformers-0.2.1/./ktransformers/util/utils.py", line 83, in load_weights
load_cur_state_dict(module, gguf_loader, prefix)
File "/host/ktransformers-0.2.1/./ktransformers/util/utils.py", line 78, in load_cur_state_dict
raise Exception(f"can't find {translated_key} in GGUF file!")
Exception: can't find model.tok_embeddings.weight in GGUF file!
`

Originally created by @4thGardenOfQMH on GitHub (Feb 23, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/616 Originally assigned to: @Azure-Tang, @ovowei, @Atream on GitHub. 我试图使用命令 python ./ktransformers/local_chat.py --model_path /host/internlm2_5-7b-chat-1m --gguf_path /host/internlm2_5-7b-chat-1m-gguf --cpu_infer 33 --max_new_tokens 1000 --mode "/***/ktransformers-0.2.1/ktransformers/optimize/optimize_rules/Internlm2_5-7b-Chat-1m.yaml"启动internlm2_5-7b-chat-1m模型，使用internlm2_5-7b-chat-1m-q4_k_m.gguf，下载的是上海人工智能实验室提供的量化后版本，但是遇到了如下报错，请问是模型有问题，还是不支持呢？如果还需要什么信息，请告诉我，多谢！ ` You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour please input the path of your rule file(yaml file containing optimize rules):/host/ktransformers-0.2.1/ktransformers/optimize/optimize_rules/Internlm2_5-7b-Chat-1m.yaml Injecting model as default Injecting model.tok_embeddings as default Injecting model.layers as default Injecting model.layers.0 as default Injecting model.layers.0.attention as default Injecting model.layers.0.attention.wqkv as default Injecting model.layers.0.attention.wo as default Injecting model.layers.0.attention.rotary_emb as default Injecting model.layers.0.feed_forward as default Injecting model.layers.0.feed_forward.w1 as default Injecting model.layers.0.feed_forward.w3 as default Injecting model.layers.0.feed_forward.w2 as default Injecting model.layers.0.feed_forward.act_fn as default Injecting model.layers.0.attention_norm as default Injecting model.layers.0.ffn_norm as default Injecting model.layers.1 as default Injecting model.layers.1.attention as default Injecting model.layers.1.attention.wqkv as default Injecting model.layers.1.attention.wo as default Injecting model.layers.1.attention.rotary_emb as default Injecting model.layers.1.feed_forward as default Injecting model.layers.1.feed_forward.w1 as default Injecting model.layers.1.feed_forward.w3 as default Injecting model.layers.1.feed_forward.w2 as default Injecting model.layers.1.feed_forward.act_fn as default Injecting model.layers.1.attention_norm as default Injecting model.layers.1.ffn_norm as default Injecting model.layers.2 as default Injecting model.layers.2.attention as default Injecting model.layers.2.attention.wqkv as default Injecting model.layers.2.attention.wo as default Injecting model.layers.2.attention.rotary_emb as default Injecting model.layers.2.feed_forward as default Injecting model.layers.2.feed_forward.w1 as default Injecting model.layers.2.feed_forward.w3 as default Injecting model.layers.2.feed_forward.w2 as default Injecting model.layers.2.feed_forward.act_fn as default Injecting model.layers.2.attention_norm as default Injecting model.layers.2.ffn_norm as default Injecting model.layers.3 as default Injecting model.layers.3.attention as default Injecting model.layers.3.attention.wqkv as default Injecting model.layers.3.attention.wo as default Injecting model.layers.3.attention.rotary_emb as default Injecting model.layers.3.feed_forward as default Injecting model.layers.3.feed_forward.w1 as default Injecting model.layers.3.feed_forward.w3 as default Injecting model.layers.3.feed_forward.w2 as default Injecting model.layers.3.feed_forward.act_fn as default Injecting model.layers.3.attention_norm as default Injecting model.layers.3.ffn_norm as default Injecting model.layers.4 as default Injecting model.layers.4.attention as default Injecting model.layers.4.attention.wqkv as default Injecting model.layers.4.attention.wo as default Injecting model.layers.4.attention.rotary_emb as default Injecting model.layers.4.feed_forward as default Injecting model.layers.4.feed_forward.w1 as default Injecting model.layers.4.feed_forward.w3 as default Injecting model.layers.4.feed_forward.w2 as default Injecting model.layers.4.feed_forward.act_fn as default Injecting model.layers.4.attention_norm as default Injecting model.layers.4.ffn_norm as default Injecting model.layers.5 as default Injecting model.layers.5.attention as default Injecting model.layers.5.attention.wqkv as default Injecting model.layers.5.attention.wo as default Injecting model.layers.5.attention.rotary_emb as default Injecting model.layers.5.feed_forward as default Injecting model.layers.5.feed_forward.w1 as default Injecting model.layers.5.feed_forward.w3 as default Injecting model.layers.5.feed_forward.w2 as default Injecting model.layers.5.feed_forward.act_fn as default Injecting model.layers.5.attention_norm as default Injecting model.layers.5.ffn_norm as default Injecting model.layers.6 as default Injecting model.layers.6.attention as default Injecting model.layers.6.attention.wqkv as default Injecting model.layers.6.attention.wo as default Injecting model.layers.6.attention.rotary_emb as default Injecting model.layers.6.feed_forward as default Injecting model.layers.6.feed_forward.w1 as default Injecting model.layers.6.feed_forward.w3 as default Injecting model.layers.6.feed_forward.w2 as default Injecting model.layers.6.feed_forward.act_fn as default Injecting model.layers.6.attention_norm as default Injecting model.layers.6.ffn_norm as default Injecting model.layers.7 as default Injecting model.layers.7.attention as default Injecting model.layers.7.attention.wqkv as default Injecting model.layers.7.attention.wo as default Injecting model.layers.7.attention.rotary_emb as default Injecting model.layers.7.feed_forward as default Injecting model.layers.7.feed_forward.w1 as default Injecting model.layers.7.feed_forward.w3 as default Injecting model.layers.7.feed_forward.w2 as default Injecting model.layers.7.feed_forward.act_fn as default Injecting model.layers.7.attention_norm as default Injecting model.layers.7.ffn_norm as default Injecting model.layers.8 as default Injecting model.layers.8.attention as default Injecting model.layers.8.attention.wqkv as default Injecting model.layers.8.attention.wo as default Injecting model.layers.8.attention.rotary_emb as default Injecting model.layers.8.feed_forward as default Injecting model.layers.8.feed_forward.w1 as default Injecting model.layers.8.feed_forward.w3 as default Injecting model.layers.8.feed_forward.w2 as default Injecting model.layers.8.feed_forward.act_fn as default Injecting model.layers.8.attention_norm as default Injecting model.layers.8.ffn_norm as default Injecting model.layers.9 as default Injecting model.layers.9.attention as default Injecting model.layers.9.attention.wqkv as default Injecting model.layers.9.attention.wo as default Injecting model.layers.9.attention.rotary_emb as default Injecting model.layers.9.feed_forward as default Injecting model.layers.9.feed_forward.w1 as default Injecting model.layers.9.feed_forward.w3 as default Injecting model.layers.9.feed_forward.w2 as default Injecting model.layers.9.feed_forward.act_fn as default Injecting model.layers.9.attention_norm as default Injecting model.layers.9.ffn_norm as default Injecting model.layers.10 as default Injecting model.layers.10.attention as default Injecting model.layers.10.attention.wqkv as default Injecting model.layers.10.attention.wo as default Injecting model.layers.10.attention.rotary_emb as default Injecting model.layers.10.feed_forward as default Injecting model.layers.10.feed_forward.w1 as default Injecting model.layers.10.feed_forward.w3 as default Injecting model.layers.10.feed_forward.w2 as default Injecting model.layers.10.feed_forward.act_fn as default Injecting model.layers.10.attention_norm as default Injecting model.layers.10.ffn_norm as default Injecting model.layers.11 as default Injecting model.layers.11.attention as default Injecting model.layers.11.attention.wqkv as default Injecting model.layers.11.attention.wo as default Injecting model.layers.11.attention.rotary_emb as default Injecting model.layers.11.feed_forward as default Injecting model.layers.11.feed_forward.w1 as default Injecting model.layers.11.feed_forward.w3 as default Injecting model.layers.11.feed_forward.w2 as default Injecting model.layers.11.feed_forward.act_fn as default Injecting model.layers.11.attention_norm as default Injecting model.layers.11.ffn_norm as default Injecting model.layers.12 as default Injecting model.layers.12.attention as default Injecting model.layers.12.attention.wqkv as default Injecting model.layers.12.attention.wo as default Injecting model.layers.12.attention.rotary_emb as default Injecting model.layers.12.feed_forward as default Injecting model.layers.12.feed_forward.w1 as default Injecting model.layers.12.feed_forward.w3 as default Injecting model.layers.12.feed_forward.w2 as default Injecting model.layers.12.feed_forward.act_fn as default Injecting model.layers.12.attention_norm as default Injecting model.layers.12.ffn_norm as default Injecting model.layers.13 as default Injecting model.layers.13.attention as default Injecting model.layers.13.attention.wqkv as default Injecting model.layers.13.attention.wo as default Injecting model.layers.13.attention.rotary_emb as default Injecting model.layers.13.feed_forward as default Injecting model.layers.13.feed_forward.w1 as default Injecting model.layers.13.feed_forward.w3 as default Injecting model.layers.13.feed_forward.w2 as default Injecting model.layers.13.feed_forward.act_fn as default Injecting model.layers.13.attention_norm as default Injecting model.layers.13.ffn_norm as default Injecting model.layers.14 as default Injecting model.layers.14.attention as default Injecting model.layers.14.attention.wqkv as default Injecting model.layers.14.attention.wo as default Injecting model.layers.14.attention.rotary_emb as default Injecting model.layers.14.feed_forward as default Injecting model.layers.14.feed_forward.w1 as default Injecting model.layers.14.feed_forward.w3 as default Injecting model.layers.14.feed_forward.w2 as default Injecting model.layers.14.feed_forward.act_fn as default Injecting model.layers.14.attention_norm as default Injecting model.layers.14.ffn_norm as default Injecting model.layers.15 as default Injecting model.layers.15.attention as default Injecting model.layers.15.attention.wqkv as default Injecting model.layers.15.attention.wo as default Injecting model.layers.15.attention.rotary_emb as default Injecting model.layers.15.feed_forward as default Injecting model.layers.15.feed_forward.w1 as default Injecting model.layers.15.feed_forward.w3 as default Injecting model.layers.15.feed_forward.w2 as default Injecting model.layers.15.feed_forward.act_fn as default Injecting model.layers.15.attention_norm as default Injecting model.layers.15.ffn_norm as default Injecting model.layers.16 as default Injecting model.layers.16.attention as default Injecting model.layers.16.attention.wqkv as default Injecting model.layers.16.attention.wo as default Injecting model.layers.16.attention.rotary_emb as default Injecting model.layers.16.feed_forward as default Injecting model.layers.16.feed_forward.w1 as default Injecting model.layers.16.feed_forward.w3 as default Injecting model.layers.16.feed_forward.w2 as default Injecting model.layers.16.feed_forward.act_fn as default Injecting model.layers.16.attention_norm as default Injecting model.layers.16.ffn_norm as default Injecting model.layers.17 as default Injecting model.layers.17.attention as default Injecting model.layers.17.attention.wqkv as default Injecting model.layers.17.attention.wo as default Injecting model.layers.17.attention.rotary_emb as default Injecting model.layers.17.feed_forward as default Injecting model.layers.17.feed_forward.w1 as default Injecting model.layers.17.feed_forward.w3 as default Injecting model.layers.17.feed_forward.w2 as default Injecting model.layers.17.feed_forward.act_fn as default Injecting model.layers.17.attention_norm as default Injecting model.layers.17.ffn_norm as default Injecting model.layers.18 as default Injecting model.layers.18.attention as default Injecting model.layers.18.attention.wqkv as default Injecting model.layers.18.attention.wo as default Injecting model.layers.18.attention.rotary_emb as default Injecting model.layers.18.feed_forward as default Injecting model.layers.18.feed_forward.w1 as default Injecting model.layers.18.feed_forward.w3 as default Injecting model.layers.18.feed_forward.w2 as default Injecting model.layers.18.feed_forward.act_fn as default Injecting model.layers.18.attention_norm as default Injecting model.layers.18.ffn_norm as default Injecting model.layers.19 as default Injecting model.layers.19.attention as default Injecting model.layers.19.attention.wqkv as default Injecting model.layers.19.attention.wo as default Injecting model.layers.19.attention.rotary_emb as default Injecting model.layers.19.feed_forward as default Injecting model.layers.19.feed_forward.w1 as default Injecting model.layers.19.feed_forward.w3 as default Injecting model.layers.19.feed_forward.w2 as default Injecting model.layers.19.feed_forward.act_fn as default Injecting model.layers.19.attention_norm as default Injecting model.layers.19.ffn_norm as default Injecting model.layers.20 as default Injecting model.layers.20.attention as default Injecting model.layers.20.attention.wqkv as default Injecting model.layers.20.attention.wo as default Injecting model.layers.20.attention.rotary_emb as default Injecting model.layers.20.feed_forward as default Injecting model.layers.20.feed_forward.w1 as default Injecting model.layers.20.feed_forward.w3 as default Injecting model.layers.20.feed_forward.w2 as default Injecting model.layers.20.feed_forward.act_fn as default Injecting model.layers.20.attention_norm as default Injecting model.layers.20.ffn_norm as default Injecting model.layers.21 as default Injecting model.layers.21.attention as default Injecting model.layers.21.attention.wqkv as default Injecting model.layers.21.attention.wo as default Injecting model.layers.21.attention.rotary_emb as default Injecting model.layers.21.feed_forward as default Injecting model.layers.21.feed_forward.w1 as default Injecting model.layers.21.feed_forward.w3 as default Injecting model.layers.21.feed_forward.w2 as default Injecting model.layers.21.feed_forward.act_fn as default Injecting model.layers.21.attention_norm as default Injecting model.layers.21.ffn_norm as default Injecting model.layers.22 as default Injecting model.layers.22.attention as default Injecting model.layers.22.attention.wqkv as default Injecting model.layers.22.attention.wo as default Injecting model.layers.22.attention.rotary_emb as default Injecting model.layers.22.feed_forward as default Injecting model.layers.22.feed_forward.w1 as default Injecting model.layers.22.feed_forward.w3 as default Injecting model.layers.22.feed_forward.w2 as default Injecting model.layers.22.feed_forward.act_fn as default Injecting model.layers.22.attention_norm as default Injecting model.layers.22.ffn_norm as default Injecting model.layers.23 as default Injecting model.layers.23.attention as default Injecting model.layers.23.attention.wqkv as default Injecting model.layers.23.attention.wo as default Injecting model.layers.23.attention.rotary_emb as default Injecting model.layers.23.feed_forward as default Injecting model.layers.23.feed_forward.w1 as default Injecting model.layers.23.feed_forward.w3 as default Injecting model.layers.23.feed_forward.w2 as default Injecting model.layers.23.feed_forward.act_fn as default Injecting model.layers.23.attention_norm as default Injecting model.layers.23.ffn_norm as default Injecting model.layers.24 as default Injecting model.layers.24.attention as default Injecting model.layers.24.attention.wqkv as default Injecting model.layers.24.attention.wo as default Injecting model.layers.24.attention.rotary_emb as default Injecting model.layers.24.feed_forward as default Injecting model.layers.24.feed_forward.w1 as default Injecting model.layers.24.feed_forward.w3 as default Injecting model.layers.24.feed_forward.w2 as default Injecting model.layers.24.feed_forward.act_fn as default Injecting model.layers.24.attention_norm as default Injecting model.layers.24.ffn_norm as default Injecting model.layers.25 as default Injecting model.layers.25.attention as default Injecting model.layers.25.attention.wqkv as default Injecting model.layers.25.attention.wo as default Injecting model.layers.25.attention.rotary_emb as default Injecting model.layers.25.feed_forward as default Injecting model.layers.25.feed_forward.w1 as default Injecting model.layers.25.feed_forward.w3 as default Injecting model.layers.25.feed_forward.w2 as default Injecting model.layers.25.feed_forward.act_fn as default Injecting model.layers.25.attention_norm as default Injecting model.layers.25.ffn_norm as default Injecting model.layers.26 as default Injecting model.layers.26.attention as default Injecting model.layers.26.attention.wqkv as default Injecting model.layers.26.attention.wo as default Injecting model.layers.26.attention.rotary_emb as default Injecting model.layers.26.feed_forward as default Injecting model.layers.26.feed_forward.w1 as default Injecting model.layers.26.feed_forward.w3 as default Injecting model.layers.26.feed_forward.w2 as default Injecting model.layers.26.feed_forward.act_fn as default Injecting model.layers.26.attention_norm as default Injecting model.layers.26.ffn_norm as default Injecting model.layers.27 as default Injecting model.layers.27.attention as default Injecting model.layers.27.attention.wqkv as default Injecting model.layers.27.attention.wo as default Injecting model.layers.27.attention.rotary_emb as default Injecting model.layers.27.feed_forward as default Injecting model.layers.27.feed_forward.w1 as default Injecting model.layers.27.feed_forward.w3 as default Injecting model.layers.27.feed_forward.w2 as default Injecting model.layers.27.feed_forward.act_fn as default Injecting model.layers.27.attention_norm as default Injecting model.layers.27.ffn_norm as default Injecting model.layers.28 as default Injecting model.layers.28.attention as default Injecting model.layers.28.attention.wqkv as default Injecting model.layers.28.attention.wo as default Injecting model.layers.28.attention.rotary_emb as default Injecting model.layers.28.feed_forward as default Injecting model.layers.28.feed_forward.w1 as default Injecting model.layers.28.feed_forward.w3 as default Injecting model.layers.28.feed_forward.w2 as default Injecting model.layers.28.feed_forward.act_fn as default Injecting model.layers.28.attention_norm as default Injecting model.layers.28.ffn_norm as default Injecting model.layers.29 as default Injecting model.layers.29.attention as default Injecting model.layers.29.attention.wqkv as default Injecting model.layers.29.attention.wo as default Injecting model.layers.29.attention.rotary_emb as default Injecting model.layers.29.feed_forward as default Injecting model.layers.29.feed_forward.w1 as default Injecting model.layers.29.feed_forward.w3 as default Injecting model.layers.29.feed_forward.w2 as default Injecting model.layers.29.feed_forward.act_fn as default Injecting model.layers.29.attention_norm as default Injecting model.layers.29.ffn_norm as default Injecting model.layers.30 as default Injecting model.layers.30.attention as default Injecting model.layers.30.attention.wqkv as default Injecting model.layers.30.attention.wo as default Injecting model.layers.30.attention.rotary_emb as default Injecting model.layers.30.feed_forward as default Injecting model.layers.30.feed_forward.w1 as default Injecting model.layers.30.feed_forward.w3 as default Injecting model.layers.30.feed_forward.w2 as default Injecting model.layers.30.feed_forward.act_fn as default Injecting model.layers.30.attention_norm as default Injecting model.layers.30.ffn_norm as default Injecting model.layers.31 as default Injecting model.layers.31.attention as default Injecting model.layers.31.attention.wqkv as default Injecting model.layers.31.attention.wo as default Injecting model.layers.31.attention.rotary_emb as default Injecting model.layers.31.feed_forward as default Injecting model.layers.31.feed_forward.w1 as default Injecting model.layers.31.feed_forward.w3 as default Injecting model.layers.31.feed_forward.w2 as default Injecting model.layers.31.feed_forward.act_fn as default Injecting model.layers.31.attention_norm as default Injecting model.layers.31.ffn_norm as default Injecting model.norm as default Injecting output as default Traceback (most recent call last): File "/host/ktransformers-0.2.1/./ktransformers/local_chat.py", line 179, in <module> fire.Fire(local_chat) File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/host/ktransformers-0.2.1/./ktransformers/local_chat.py", line 110, in local_chat optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config) File "/host/ktransformers-0.2.1/./ktransformers/optimize/optimize.py", line 129, in optimize_and_load_gguf load_weights(module, gguf_loader) File "/host/ktransformers-0.2.1/./ktransformers/util/utils.py", line 85, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/host/ktransformers-0.2.1/./ktransformers/util/utils.py", line 85, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/host/ktransformers-0.2.1/./ktransformers/util/utils.py", line 83, in load_weights load_cur_state_dict(module, gguf_loader, prefix) File "/host/ktransformers-0.2.1/./ktransformers/util/utils.py", line 78, in load_cur_state_dict raise Exception(f"can't find {translated_key} in GGUF file!") Exception: can't find model.tok_embeddings.weight in GGUF file! `
bsakaguchi commented 2026-10-29 14:39:55 +00:00
Author
Owner
Copy link

@ubergarm commented on GitHub (Feb 23, 2025):

同学，可以查看这边关于如何运行小规模密集型模型（例如「internlm2_5-7b」）的技术问题。

https://github.com/kvcache-ai/ktransformers/issues/611#issuecomment-2676984590

Hey bud, check out this issue over here for running small dense models like internlm2_5-7b on other inference engines:

<!-- gh-comment-id:2676996005 --> @ubergarm commented on GitHub (Feb 23, 2025): 同学，可以查看这边关于如何运行小规模密集型模型（例如「internlm2_5-7b」）的技术问题。 https://github.com/kvcache-ai/ktransformers/issues/611#issuecomment-2676984590 --- Hey bud, check out this issue over here for running small dense models like `internlm2_5-7b` on other inference engines:
bsakaguchi commented 2026-10-29 14:39:55 +00:00
Author
Owner
Copy link

@ayttop commented on GitHub (Jun 15, 2025):

mixtral 8*7b run or no with
ktransformers???

<!-- gh-comment-id:2974653686 --> @ayttop commented on GitHub (Jun 15, 2025): mixtral 8*7b run or no with ktransformers???
bsakaguchi commented 2026-10-29 14:39:55 +00:00
Author
Owner
Copy link

@ayttop commented on GitHub (Jun 15, 2025):

Moonlight?????

Qwen2-57B-A14B?????

it suport in ktransformers?????

<!-- gh-comment-id:2974658924 --> @ayttop commented on GitHub (Jun 15, 2025): Moonlight????? Qwen2-57B-A14B????? it suport in ktransformers?????
bsakaguchi commented 2026-10-29 14:39:56 +00:00
Author
Owner
Copy link

@ayttop commented on GitHub (Jun 15, 2025):

If anyone has tried or knows,
Moonlight?????

Qwen2-57B-A14B?????
mixtral 8*7b ؟؟؟؟

there is a Yaml for them, but there is a Yaml called Internlm2_5-7b-Chat-1m.yaml, and it doesn't work, as far as I know.

<!-- gh-comment-id:2974661349 --> @ayttop commented on GitHub (Jun 15, 2025): If anyone has tried or knows, Moonlight????? Qwen2-57B-A14B????? mixtral 8*7b ؟؟؟؟ there is a Yaml for them, but there is a Yaml called Internlm2_5-7b-Chat-1m.yaml, and it doesn't work, as far as I know.
bsakaguchi commented 2026-10-29 14:39:56 +00:00
Author
Owner
Copy link

@ayttop commented on GitHub (Jun 15, 2025):

I want to try the library, not the models.
I can't load large models.

<!-- gh-comment-id:2974661995 --> @ayttop commented on GitHub (Jun 15, 2025): I want to try the library, not the models. I can't load large models.
bsakaguchi commented 2026-10-29 14:39:56 +00:00
Author
Owner
Copy link

@ayttop commented on GitHub (Jun 15, 2025):

https://github.com/kvcache-ai/ktransformers/issues/1390

<!-- gh-comment-id:2974662591 --> @ayttop commented on GitHub (Jun 15, 2025): https://github.com/kvcache-ai/ktransformers/issues/1390
bsakaguchi commented 2026-10-29 14:39:56 +00:00
Author
Owner
Copy link

@Atream commented on GitHub (Jun 16, 2025):

We previously supported local inference of InternLM's 1M context, but the model's support has been damaged with the version update. We'll fix it asap.

<!-- gh-comment-id:2974938769 --> @Atream commented on GitHub (Jun 16, 2025): We previously supported local inference of InternLM's 1M context, but the model's support has been damaged with the version update. We'll fix it asap.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#456
No description provided.