kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #289] ktransformers ubuntu 24.04. 运行失败 #186

New issue
Open
opened 2026-10-29 14:35:29 +00:00 by bsakaguchi · 11 comments
bsakaguchi commented 2026-10-29 14:35:29 +00:00
Owner
Copy link

Originally created by @bluezd on GitHub (Feb 14, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/289

环境

Ubuntu 24.04
Driver Version: 570.86.10 CUDA Version: 12.8
宿主机 CPU：Intel(R) Xeon(R) Gold 5218R CPU @ 2.10GHz
虚拟机 CPU:

flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault pti ssbd ibrs ibpb fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 arat pku ospke avx512_vnni

错误 1

在虚拟机中测试模型报如下错误：

python -m ktransformers.local_chat --model_path deepseek-ai/DeepSeek-R1 --gguf_path ./DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF/
using custom modeling_xxx.py.
using default_optimize_rule for DeepseekV3ForCausalLM
Injecting model as ktransformers.operators.models . KDeepseekV2Model
Injecting model.embed_tokens as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.0.self_attn.q_a_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.self_attn.q_a_layernorm as default
Injecting model.layers.0.self_attn.q_b_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.self_attn.kv_a_layernorm as default
Injecting model.layers.0.self_attn.kv_b_proj as default
Injecting model.layers.0.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbeddingV3
...
Injecting model.layers.60.post_attention_layernorm as default
Injecting model.norm as default
Injecting lm_head as default
loading token_embd.weight to cpu
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/root/ktransformers/ktransformers/local_chat.py", line 179, in <module>
    fire.Fire(local_chat)
  File "/root/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/root/ktransformers/ktransformers/local_chat.py", line 110, in local_chat
    optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config)
  File "/root/ktransformers/ktransformers/optimize/optimize.py", line 129, in optimize_and_load_gguf
    load_weights(module, gguf_loader)
  File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights
    load_weights(child, gguf_loader, prefix+name+".")
  File "/root/ktransformers/ktransformers/util/utils.py", line 85, in load_weights
    module.load()
  File "/root/ktransformers/ktransformers/operators/base_operator.py", line 60, in load
    utils.load_weights(child, self.gguf_loader, self.key+".")
  File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights
    load_weights(child, gguf_loader, prefix+name+".")
  File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights
    load_weights(child, gguf_loader, prefix+name+".")
  File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights
    load_weights(child, gguf_loader, prefix+name+".")
  File "/root/ktransformers/ktransformers/util/utils.py", line 85, in load_weights
    module.load()
  File "/root/ktransformers/ktransformers/operators/base_operator.py", line 60, in load
    utils.load_weights(child, self.gguf_loader, self.key+".")
  File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights
    load_weights(child, gguf_loader, prefix+name+".")
  File "/root/ktransformers/ktransformers/util/utils.py", line 85, in load_weights
    module.load()
  File "/root/ktransformers/ktransformers/operators/linear.py", line 430, in load
    self.generate_linear.load(w=w)
  File "/root/ktransformers/ktransformers/operators/linear.py", line 198, in load
    w = self.load_weight(device=device)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/ktransformers/ktransformers/operators/linear.py", line 92, in load_weight
    raise FileNotFoundError(f"Weight file not found for key {key}")
FileNotFoundError: Weight file not found for key blk.0.attn_q_a

模型连接：https://hf-mirror.com/Donnyed/DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF/blob/main/deepseek-r1-distill-qwen-32b-q4_k_m.gguf

错误2

参考文档：https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/DeepseekR1_V3_tutorial.md#settings-1
执行: pip install ./ktransformers-0.3.0rc0+cu126torch26fancy-cp311-cp311-linux_x86_64.whl 之后报如下错误：

(ktransformers) root@LLM-inference:~/ktransformers# python -m ktransformers.local_chat --model_path deepseek-ai/DeepSeek-R1 --gguf_path ./DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF/
using custom modeling_xxx.py.
using default_optimize_rule for DeepseekV3ForCausalLM
__AVX512F__
Injecting model as ktransformers.operators.models . KDeepseekV2Model
Injecting model.embed_tokens as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Illegal instruction (core dumped)

错误3

手动安装包，报如下错误：

# pip install ./ktransformers-0.2.0+cu121torch23avx2-cp310-cp310-linux_x86_64.whl
ERROR: ktransformers-0.2.0+cu121torch23avx2-cp310-cp310-linux_x86_64.whl is not a supported wheel on this platform.
Originally created by @bluezd on GitHub (Feb 14, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/289 ## 环境 Ubuntu 24.04 Driver Version: 570.86.10 CUDA Version: 12.8 宿主机 CPU：Intel(R) Xeon(R) Gold 5218R CPU @ 2.10GHz 虚拟机 CPU: ``` flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault pti ssbd ibrs ibpb fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 arat pku ospke avx512_vnni ``` ## 错误 1 在虚拟机中测试模型报如下错误： ``` python -m ktransformers.local_chat --model_path deepseek-ai/DeepSeek-R1 --gguf_path ./DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF/ using custom modeling_xxx.py. using default_optimize_rule for DeepseekV3ForCausalLM Injecting model as ktransformers.operators.models . KDeepseekV2Model Injecting model.embed_tokens as default Injecting model.layers as default Injecting model.layers.0 as default Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention Injecting model.layers.0.self_attn.q_a_proj as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.0.self_attn.q_a_layernorm as default Injecting model.layers.0.self_attn.q_b_proj as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.0.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.0.self_attn.kv_a_layernorm as default Injecting model.layers.0.self_attn.kv_b_proj as default Injecting model.layers.0.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.0.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbeddingV3 ... Injecting model.layers.60.post_attention_layernorm as default Injecting model.norm as default Injecting lm_head as default loading token_embd.weight to cpu Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/root/ktransformers/ktransformers/local_chat.py", line 179, in <module> fire.Fire(local_chat) File "/root/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/root/ktransformers/ktransformers/local_chat.py", line 110, in local_chat optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config) File "/root/ktransformers/ktransformers/optimize/optimize.py", line 129, in optimize_and_load_gguf load_weights(module, gguf_loader) File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/root/ktransformers/ktransformers/util/utils.py", line 85, in load_weights module.load() File "/root/ktransformers/ktransformers/operators/base_operator.py", line 60, in load utils.load_weights(child, self.gguf_loader, self.key+".") File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/root/ktransformers/ktransformers/util/utils.py", line 85, in load_weights module.load() File "/root/ktransformers/ktransformers/operators/base_operator.py", line 60, in load utils.load_weights(child, self.gguf_loader, self.key+".") File "/root/ktransformers/ktransformers/util/utils.py", line 83, in load_weights load_weights(child, gguf_loader, prefix+name+".") File "/root/ktransformers/ktransformers/util/utils.py", line 85, in load_weights module.load() File "/root/ktransformers/ktransformers/operators/linear.py", line 430, in load self.generate_linear.load(w=w) File "/root/ktransformers/ktransformers/operators/linear.py", line 198, in load w = self.load_weight(device=device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/ktransformers/ktransformers/operators/linear.py", line 92, in load_weight raise FileNotFoundError(f"Weight file not found for key {key}") FileNotFoundError: Weight file not found for key blk.0.attn_q_a ``` 模型连接：https://hf-mirror.com/Donnyed/DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF/blob/main/deepseek-r1-distill-qwen-32b-q4_k_m.gguf ## 错误2 参考文档：https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/DeepseekR1_V3_tutorial.md#settings-1 执行: `pip install ./ktransformers-0.3.0rc0+cu126torch26fancy-cp311-cp311-linux_x86_64.whl` 之后报如下错误： ``` (ktransformers) root@LLM-inference:~/ktransformers# python -m ktransformers.local_chat --model_path deepseek-ai/DeepSeek-R1 --gguf_path ./DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF/ using custom modeling_xxx.py. using default_optimize_rule for DeepseekV3ForCausalLM __AVX512F__ Injecting model as ktransformers.operators.models . KDeepseekV2Model Injecting model.embed_tokens as default Injecting model.layers as default Injecting model.layers.0 as default Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention Illegal instruction (core dumped) ``` ## 错误3 手动安装包，报如下错误： ``` # pip install ./ktransformers-0.2.0+cu121torch23avx2-cp310-cp310-linux_x86_64.whl ERROR: ktransformers-0.2.0+cu121torch23avx2-cp310-cp310-linux_x86_64.whl is not a supported wheel on this platform. ```
bsakaguchi commented 2026-10-29 14:35:30 +00:00
Author
Owner
Copy link

@bennyyu79 commented on GitHub (Feb 15, 2025):

python -m ktransformers.local_chat --model_path /home/work/tools/llm/DeepSeek-R1 --gguf_path /home/work/.cache/huggingface/hub/models--unsloth--DeepSeek-R1-GGUF/snapshots/02bcc0a0f68146dae57942804d82bdf0cc636003/DeepSeek-R1-Q4_K_M --prompt_file test_prompt.txt --cpu_infer 65 --max_new_tokens 1000
using custom modeling_xxx.py.
using default_optimize_rule for DeepseekV3ForCausalLM
AVX512F
Injecting model as ktransformers.operators.models . KDeepseekV2Model
Injecting model.embed_tokens as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Illegal instruction

<!-- gh-comment-id:2660850517 --> @bennyyu79 commented on GitHub (Feb 15, 2025): python -m ktransformers.local_chat --model_path /home/work/tools/llm/DeepSeek-R1 --gguf_path /home/work/.cache/huggingface/hub/models--unsloth--DeepSeek-R1-GGUF/snapshots/02bcc0a0f68146dae57942804d82bdf0cc636003/DeepSeek-R1-Q4_K_M --prompt_file test_prompt.txt --cpu_infer 65 --max_new_tokens 1000 using custom modeling_xxx.py. using default_optimize_rule for DeepseekV3ForCausalLM __AVX512F__ Injecting model as ktransformers.operators.models . KDeepseekV2Model Injecting model.embed_tokens as default Injecting model.layers as default Injecting model.layers.0 as default Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention Illegal instruction
bsakaguchi commented 2026-10-29 14:35:30 +00:00
Author
Owner
Copy link

@bluezd commented on GitHub (Feb 15, 2025):

哪位老师帮忙看下这个问题？

<!-- gh-comment-id:2660917434 --> @bluezd commented on GitHub (Feb 15, 2025): 哪位老师帮忙看下这个问题？
bsakaguchi commented 2026-10-29 14:35:30 +00:00
Author
Owner
Copy link

@Allonsy-1 commented on GitHub (Feb 16, 2025):

错误3

手动安装包，报如下错误：

# pip install ./ktransformers-0.2.0+cu121torch23avx2-cp310-cp310-linux_x86_64.whl
ERROR: ktransformers-0.2.0+cu121torch23avx2-cp310-cp310-linux_x86_64.whl is not a supported wheel on this platform.

需要先创建一个虚拟环境 python要3.10 并且install pip?

<!-- gh-comment-id:2661315711 --> @Allonsy-1 commented on GitHub (Feb 16, 2025): > ## 错误3 > 手动安装包，报如下错误： > > ``` > # pip install ./ktransformers-0.2.0+cu121torch23avx2-cp310-cp310-linux_x86_64.whl > ERROR: ktransformers-0.2.0+cu121torch23avx2-cp310-cp310-linux_x86_64.whl is not a supported wheel on this platform. > ``` 需要先创建一个虚拟环境 python要3.10 并且install pip?
bsakaguchi commented 2026-10-29 14:35:30 +00:00
Author
Owner
Copy link

@1850772210 commented on GitHub (Feb 17, 2025):

你的cpu并不支持amx指令集 KT需要支持 amx指令集

<!-- gh-comment-id:2662141103 --> @1850772210 commented on GitHub (Feb 17, 2025): 你的cpu并不支持amx指令集 KT需要支持 amx指令集
bsakaguchi commented 2026-10-29 14:35:30 +00:00
Author
Owner
Copy link

@aisfx commented on GitHub (Feb 18, 2025):

第一个问题是否解决？ 我也遇到了相同的问题

<!-- gh-comment-id:2664385404 --> @aisfx commented on GitHub (Feb 18, 2025): 第一个问题是否解决？ 我也遇到了相同的问题
bsakaguchi commented 2026-10-29 14:35:30 +00:00
Author
Owner
Copy link

@quanliu1991 commented on GitHub (Feb 18, 2025):

你的cpu并不支持amx指令集 KT需要支持 amx指令集

不支持amx指令集的CPU，有其他办法运行KT吗？比如都在GPU上计算。

<!-- gh-comment-id:2664816803 --> @quanliu1991 commented on GitHub (Feb 18, 2025): > 你的cpu并不支持amx指令集 KT需要支持 amx指令集 不支持amx指令集的CPU，有其他办法运行KT吗？比如都在GPU上计算。
bsakaguchi commented 2026-10-29 14:35:31 +00:00
Author
Owner
Copy link

@1850772210 commented on GitHub (Feb 18, 2025):

你的cpu并不支持amx指令集 KT需要支持 amx指令集

不支持amx指令集的CPU，有其他办法运行KT吗？比如都在GPU上计算。

可以手动编译0.2 版本 这种方法可以不支持amx指令集

<!-- gh-comment-id:2664829992 --> @1850772210 commented on GitHub (Feb 18, 2025): > > 你的cpu并不支持amx指令集 KT需要支持 amx指令集 > > 不支持amx指令集的CPU，有其他办法运行KT吗？比如都在GPU上计算。 可以手动编译0.2 版本 这种方法可以不支持amx指令集
bsakaguchi commented 2026-10-29 14:35:31 +00:00
Author
Owner
Copy link

@alvinfang27 commented on GitHub (Feb 19, 2025):

第一个错误我也是整了好几天，最后在另一个issue下破案了：KT不支持所有的量化模型，即你下载的DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF 这个模型运行不了

<!-- gh-comment-id:2669127496 --> @alvinfang27 commented on GitHub (Feb 19, 2025): 第一个错误我也是整了好几天，最后在另一个issue下破案了：KT不支持所有的量化模型，即你下载的DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF 这个模型运行不了
bsakaguchi commented 2026-10-29 14:35:31 +00:00
Author
Owner
Copy link

@Longleaves commented on GitHub (Feb 20, 2025):

同遇第3个问题

<!-- gh-comment-id:2670389535 --> @Longleaves commented on GitHub (Feb 20, 2025): 同遇第3个问题
bsakaguchi commented 2026-10-29 14:35:31 +00:00
Author
Owner
Copy link

@ZHUHF123 commented on GitHub (Mar 11, 2025):

第一个错误我也是整了好几天，最后在另一个issue下破案了：KT不支持所有的量化模型，即你下载的DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF 这个模型运行不了

请问在不支持amx指令集的情况下我可以用哪个模型呢，希望能得到回复

<!-- gh-comment-id:2712448083 --> @ZHUHF123 commented on GitHub (Mar 11, 2025): > 第一个错误我也是整了好几天，最后在另一个issue下破案了：KT不支持所有的量化模型，即你下载的DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF 这个模型运行不了 请问在不支持amx指令集的情况下我可以用哪个模型呢，希望能得到回复
bsakaguchi commented 2026-10-29 14:35:32 +00:00
Author
Owner
Copy link

@XiongjieDai commented on GitHub (May 22, 2025):

第一个错误我也是整了好几天，最后在另一个issue下破案了：KT不支持所有的量化模型，即你下载的DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF 这个模型运行不了

哪个issue啊

<!-- gh-comment-id:2901240345 --> @XiongjieDai commented on GitHub (May 22, 2025): > 第一个错误我也是整了好几天，最后在另一个issue下破案了：KT不支持所有的量化模型，即你下载的DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF 这个模型运行不了 哪个issue啊
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#186
No description provided.