[GH-ISSUE #850] [Bug] 部署 r1 server 成功后，请求接口，一直无法返回响应 #611

New issue

Closed

opened 2026-10-29 14:41:56 +00:00 by bsakaguchi · 9 comments

bsakaguchi commented 2026-10-29 14:41:56 +00:00

Owner

Copy link

Originally created by @lessmore991 on GitHub (Mar 10, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/850

Originally assigned to: @KMSorSMS on GitHub.

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
• 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
• 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

部署命令：
ktransformers --model_path "/deepseek/DeepSeek-R1"
--gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M"
--cpu_infer 82
--model_name deepseek-r1-671b --port 9093
--paged true --use_cuda_graph --no_flash_attn false
--max_new_tokens 16384 --max_response_tokens 16384 --batch_size 5
--optimize_config_path /workspace/ktransformers/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml

请求：
url：http://0.0.0.0:9093/v1/chat/completions
body:
{
"model": "deepseek-r1-671b",
"max_tokens": 4,
"stream": true,
"messages": [
{
"role": "user",
"content": "讲个笑话"
}
]
}

结果一直无法接收到响应，后台日志显示请求已接收到，但是没有其它日志：

Reproduction

ktransformers服务化部署命令

Environment

v0.2.2版本、GPU环境

Originally created by @lessmore991 on GitHub (Mar 10, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/850 Originally assigned to: @KMSorSMS on GitHub. ### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. - [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback. - [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. ### Describe the bug 部署命令： ktransformers --model_path "/deepseek/DeepSeek-R1" \ --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" \ --cpu_infer 82 \ --model_name deepseek-r1-671b --port 9093 \ --paged true --use_cuda_graph --no_flash_attn false \ --max_new_tokens 16384 --max_response_tokens 16384 --batch_size 5 \ --optimize_config_path /workspace/ktransformers/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml 请求： url：http://0.0.0.0:9093/v1/chat/completions body: { "model": "deepseek-r1-671b", "max_tokens": 4, "stream": true, "messages": [ { "role": "user", "content": "讲个笑话" } ] } 结果一直无法接收到响应，后台日志显示请求已接收到，但是没有其它日志：  ### Reproduction ktransformers服务化部署命令 ### Environment v0.2.2版本、GPU环境

bsakaguchi closed this issue 2026-10-29 14:41:56 +00:00

bsakaguchi commented 2026-10-29 14:41:57 +00:00

Author

Owner

Copy link

@qiuqiu10 commented on GitHub (Mar 10, 2025):

@KMSorSMS

<!-- gh-comment-id:2710570803 --> @qiuqiu10 commented on GitHub (Mar 10, 2025): @KMSorSMS

bsakaguchi commented 2026-10-29 14:41:57 +00:00

Author

Owner

Copy link

@KMSorSMS commented on GitHub (Mar 10, 2025):

Checklist

• 1. I have searched related issues but cannot get the expected help.[x] 2. The bug has not been fixed in the latest version.[x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.[x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.[x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

部署命令： ktransformers --model_path "/deepseek/DeepSeek-R1" --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" --cpu_infer 82 --model_name deepseek-r1-671b --port 9093 --paged true --use_cuda_graph --no_flash_attn false --max_new_tokens 16384 --max_response_tokens 16384 --batch_size 5 --optimize_config_path /workspace/ktransformers/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml

请求： url：http://0.0.0.0:9093/v1/chat/completions body: { "model": "deepseek-r1-671b", "max_tokens": 4, "stream": true, "messages": [ { "role": "user", "content": "讲个笑话" } ] }

结果一直无法接收到响应，后台日志显示请求已接收到，但是没有其它日志：

Reproduction

ktransformers服务化部署命令

Environment

v0.2.2版本、GPU环境

可能是你cpu_infer设置高了，你的cpu有多少核？超过物理核数上下文开销以及过多线程同步的开销会导致输出很慢。你改小之后再多花时间等等

<!-- gh-comment-id:2710614927 --> @KMSorSMS commented on GitHub (Mar 10, 2025): > ### Checklist > * [x] 1. I have searched related issues but cannot get the expected help.[x] 2. The bug has not been fixed in the latest version.[x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.[x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.[x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. > > ### Describe the bug > 部署命令： ktransformers --model_path "/deepseek/DeepSeek-R1" --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" --cpu_infer 82 --model_name deepseek-r1-671b --port 9093 --paged true --use_cuda_graph --no_flash_attn false --max_new_tokens 16384 --max_response_tokens 16384 --batch_size 5 --optimize_config_path /workspace/ktransformers/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml > > 请求： url：http://0.0.0.0:9093/v1/chat/completions body: { "model": "deepseek-r1-671b", "max_tokens": 4, "stream": true, "messages": [ { "role": "user", "content": "讲个笑话" } ] } > > 结果一直无法接收到响应，后台日志显示请求已接收到，但是没有其它日志：  > > ### Reproduction > ktransformers服务化部署命令 > > ### Environment > v0.2.2版本、GPU环境 可能是你cpu_infer设置高了，你的cpu有多少核？超过物理核数上下文开销以及过多线程同步的开销会导致输出很慢。你改小之后再多花时间等等

bsakaguchi commented 2026-10-29 14:41:57 +00:00

Author

Owner

Copy link

@ubergarm commented on GitHub (Mar 10, 2025):

@lessmore991

--batch_size 5 这个参数是从哪里来的？我的理解是，如果使用 CUDA 图（CUDA graphs），应该不需要调整这个参数。

可以尝试使用已知有效的命令，例如这份非官方指南中的示例。

另外，你构建或安装的版本是什么？可通过以下命令检查：

1. git rev-parse --short HEAD
2. pip freeze | grep ktrans

如果问题解决，请告知我们。若满意请关闭此 issue。

---

lessmore991

--batch_size 5 where did you get this from? My impression was to use cuda graphs you have to leave that alone.

Try a known working command such as the one in this unofficial guide

Also what version did you build or install? You can check with git rev-parse --short HEAD or pip freeze | grep ktrans.

Let us know if you get it working and please close this issue if you are satisfied.

<!-- gh-comment-id:2711589733 --> @ubergarm commented on GitHub (Mar 10, 2025): @lessmore991 `--batch_size 5` 这个参数是从哪里来的？我的理解是，如果使用 CUDA 图（CUDA graphs），应该不需要调整这个参数。 可以尝试使用已知有效的命令，例如[这份非官方指南中的示例](https://github.com/ubergarm/r1-ktransformers-guide/blob/main/README.zh.md)。 另外，你构建或安装的版本是什么？可通过以下命令检查： 1. `git rev-parse --short HEAD` 2. `pip freeze | grep ktrans` 如果问题解决，请告知我们。若满意请关闭此 issue。 --- lessmore991 `--batch_size 5 ` where did you get this from? My impression was to use cuda graphs you have to leave that alone. Try a known working command such as [the one in this unofficial guide](https://github.com/ubergarm/r1-ktransformers-guide/blob/main/README.zh.md) Also what version did you build or install? You can check with `git rev-parse --short HEAD` or `pip freeze | grep ktrans`. Let us know if you get it working and please close this issue if you are satisfied.

bsakaguchi commented 2026-10-29 14:41:57 +00:00

Author

Owner

Copy link

@lessmore991 commented on GitHub (Mar 11, 2025):

Checklist

• 1. I have searched related issues but cannot get the expected help.[x] 2. The bug has not been fixed in the latest version.[x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.[x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.[x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

部署命令： ktransformers --model_path "/deepseek/DeepSeek-R1" --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" --cpu_infer 82 --model_name deepseek-r1-671b --port 9093 --paged true --use_cuda_graph --no_flash_attn false --max_new_tokens 16384 --max_response_tokens 16384 --batch_size 5 --optimize_config_path /workspace/ktransformers/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml
请求： url：http://0.0.0.0:9093/v1/chat/completions body: { "model": "deepseek-r1-671b", "max_tokens": 4, "stream": true, "messages": [ { "role": "user", "content": "讲个笑话" } ] }
结果一直无法接收到响应，后台日志显示请求已接收到，但是没有其它日志：

Reproduction

ktransformers服务化部署命令

Environment

v0.2.2版本、GPU环境

可能是你cpu_infer设置高了，你的cpu有多少核？超过物理核数上下文开销以及过多线程同步的开销会导致输出很慢。你改小之后再多花时间等等

cpu2个，物理核各44个，版本是0.2.2rc2+cu121torch23avx512

<!-- gh-comment-id:2712824242 --> @lessmore991 commented on GitHub (Mar 11, 2025): > > ### Checklist > > > > > > * [x] 1. I have searched related issues but cannot get the expected help.[x] 2. The bug has not been fixed in the latest version.[x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.[x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.[x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. > > ### Describe the bug > > 部署命令： ktransformers --model_path "/deepseek/DeepSeek-R1" --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" --cpu_infer 82 --model_name deepseek-r1-671b --port 9093 --paged true --use_cuda_graph --no_flash_attn false --max_new_tokens 16384 --max_response_tokens 16384 --batch_size 5 --optimize_config_path /workspace/ktransformers/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml > > 请求： url：http://0.0.0.0:9093/v1/chat/completions body: { "model": "deepseek-r1-671b", "max_tokens": 4, "stream": true, "messages": [ { "role": "user", "content": "讲个笑话" } ] } > > 结果一直无法接收到响应，后台日志显示请求已接收到，但是没有其它日志：  > > ### Reproduction > > ktransformers服务化部署命令 > > ### Environment > > v0.2.2版本、GPU环境 > > 可能是你cpu_infer设置高了，你的cpu有多少核？超过物理核数上下文开销以及过多线程同步的开销会导致输出很慢。你改小之后再多花时间等等 cpu2个，物理核各44个，版本是0.2.2rc2+cu121torch23avx512

bsakaguchi commented 2026-10-29 14:41:57 +00:00

Author

Owner

Copy link

@lessmore991 commented on GitHub (Mar 11, 2025):

@lessmore991

--batch_size 5 这个参数是从哪里来的？我的理解是，如果使用 CUDA 图（CUDA graphs），应该不需要调整这个参数。

可以尝试使用已知有效的命令，例如这份非官方指南中的示例。

另外，你构建或安装的版本是什么？可通过以下命令检查：

1. git rev-parse --short HEAD
2. pip freeze | grep ktrans

如果问题解决，请告知我们。若满意请关闭此 issue。

lessmore991

--batch_size 5 where did you get this from? My impression was to use cuda graphs you have to leave that alone.

Try a known working command such as the one in this unofficial guide

Also what version did you build or install? You can check with git rev-parse --short HEAD or pip freeze | grep ktrans.

Let us know if you get it working and please close this issue if you are satisfied.

现在把部署命令简化了之后，可以正常推理了：
ktransformers --model_path "/deepseek/DeepSeek-R1" --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" --cpu_infer 82 --model_name deepseek-r1-671b --port 9093 --max_new_tokens 16384 --max_response_tokens 16384

等了十来分钟之后，后台日志才显示有结果生成出来，但是生成的速度很慢，大概一两分钟一个token

<!-- gh-comment-id:2712828544 --> @lessmore991 commented on GitHub (Mar 11, 2025): > [@lessmore991](https://github.com/lessmore991) > > `--batch_size 5` 这个参数是从哪里来的？我的理解是，如果使用 CUDA 图（CUDA graphs），应该不需要调整这个参数。 > > 可以尝试使用已知有效的命令，例如[这份非官方指南中的示例](https://github.com/ubergarm/r1-ktransformers-guide/blob/main/README.zh.md)。 > > 另外，你构建或安装的版本是什么？可通过以下命令检查： > > 1. `git rev-parse --short HEAD` > 2. `pip freeze | grep ktrans` > > 如果问题解决，请告知我们。若满意请关闭此 issue。 > > lessmore991 > > `--batch_size 5 ` where did you get this from? My impression was to use cuda graphs you have to leave that alone. > > Try a known working command such as [the one in this unofficial guide](https://github.com/ubergarm/r1-ktransformers-guide/blob/main/README.zh.md) > > Also what version did you build or install? You can check with `git rev-parse --short HEAD` or `pip freeze | grep ktrans`. > > Let us know if you get it working and please close this issue if you are satisfied. 现在把部署命令简化了之后，可以正常推理了： ktransformers --model_path "/deepseek/DeepSeek-R1" --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" --cpu_infer 82 --model_name deepseek-r1-671b --port 9093 --max_new_tokens 16384 --max_response_tokens 16384 等了十来分钟之后，后台日志才显示有结果生成出来，但是生成的速度很慢，大概一两分钟一个token

bsakaguchi commented 2026-10-29 14:41:58 +00:00

Author

Owner

Copy link

@KMSorSMS commented on GitHub (Mar 11, 2025):

@lessmore991
--batch_size 5 这个参数是从哪里来的？我的理解是，如果使用 CUDA 图（CUDA graphs），应该不需要调整这个参数。
可以尝试使用已知有效的命令，例如这份非官方指南中的示例。
另外，你构建或安装的版本是什么？可通过以下命令检查：

1. git rev-parse --short HEAD
2. pip freeze | grep ktrans

如果问题解决，请告知我们。若满意请关闭此 issue。
lessmore991
--batch_size 5 where did you get this from? My impression was to use cuda graphs you have to leave that alone.
Try a known working command such as the one in this unofficial guide
Also what version did you build or install? You can check with git rev-parse --short HEAD or pip freeze | grep ktrans.
Let us know if you get it working and please close this issue if you are satisfied.

现在把部署命令简化了之后，可以正常推理了： ktransformers --model_path "/deepseek/DeepSeek-R1" --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" --cpu_infer 82 --model_name deepseek-r1-671b --port 9093 --max_new_tokens 16384 --max_response_tokens 16384

等了十来分钟之后，后台日志才显示有结果生成出来，但是生成的速度很慢，大概一两分钟一个token

调整一下cpu infer的数量，你lscpu查看一下当前的核数，如果没有USE_NUMA 编译时开启功能的话，你需要设置cpu infer数量和自己单个numa上的核数相当

<!-- gh-comment-id:2713152077 --> @KMSorSMS commented on GitHub (Mar 11, 2025): > > [@lessmore991](https://github.com/lessmore991) > > `--batch_size 5` 这个参数是从哪里来的？我的理解是，如果使用 CUDA 图（CUDA graphs），应该不需要调整这个参数。 > > 可以尝试使用已知有效的命令，例如[这份非官方指南中的示例](https://github.com/ubergarm/r1-ktransformers-guide/blob/main/README.zh.md)。 > > 另外，你构建或安装的版本是什么？可通过以下命令检查： > > > > 1. `git rev-parse --short HEAD` > > 2. `pip freeze | grep ktrans` > > > > 如果问题解决，请告知我们。若满意请关闭此 issue。 > > lessmore991 > > `--batch_size 5 ` where did you get this from? My impression was to use cuda graphs you have to leave that alone. > > Try a known working command such as [the one in this unofficial guide](https://github.com/ubergarm/r1-ktransformers-guide/blob/main/README.zh.md) > > Also what version did you build or install? You can check with `git rev-parse --short HEAD` or `pip freeze | grep ktrans`. > > Let us know if you get it working and please close this issue if you are satisfied. > > 现在把部署命令简化了之后，可以正常推理了： ktransformers --model_path "/deepseek/DeepSeek-R1" --gguf_path "/deepseek/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M" --cpu_infer 82 --model_name deepseek-r1-671b --port 9093 --max_new_tokens 16384 --max_response_tokens 16384 > > 等了十来分钟之后，后台日志才显示有结果生成出来，但是生成的速度很慢，大概一两分钟一个token 调整一下cpu infer的数量，你lscpu查看一下当前的核数，如果没有USE_NUMA 编译时开启功能的话，你需要设置cpu infer数量和自己单个numa上的核数相当

bsakaguchi commented 2026-10-29 14:41:58 +00:00

Author

Owner

Copy link

@lessmore991 commented on GitHub (Mar 11, 2025):

你需要设置cpu infer数量和自己单个numa上的核数相当

/workspace# lscpu | grep numa -i
NUMA node(s): 2
NUMA node0 CPU(s): 0-87
NUMA node1 CPU(s): 88-175

感谢大佬，我查看了一下配置信息如上所示，那请问是需要将cpu infer数量设置成87还是175，我现在设置的84，略小于一个numa node的核心数

<!-- gh-comment-id:2713334820 --> @lessmore991 commented on GitHub (Mar 11, 2025): > 你需要设置cpu infer数量和自己单个numa上的核数相当 /workspace# lscpu | grep numa -i NUMA node(s): 2 NUMA node0 CPU(s): 0-87 NUMA node1 CPU(s): 88-175 感谢大佬，我查看了一下配置信息如上所示，那请问是需要将cpu infer数量设置成87还是175，我现在设置的84，略小于一个numa node的核心数

bsakaguchi commented 2026-10-29 14:41:58 +00:00

Author

Owner

Copy link

@KMSorSMS commented on GitHub (Mar 11, 2025):

你需要设置cpu infer数量和自己单个numa上的核数相当

/workspace# lscpu | grep numa -i NUMA node(s): 2 NUMA node0 CPU(s): 0-87 NUMA node1 CPU(s): 88-175

感谢大佬，我查看了一下配置信息如上所示，那请问是需要将cpu infer数量设置成87还是175，我现在设置的84，略小于一个numa node的核心数

87合适，你编译的时候没有export USE_NUMA=1就行。另外，max_response_tokens这个参数应该是无效的

<!-- gh-comment-id:2713899443 --> @KMSorSMS commented on GitHub (Mar 11, 2025): > > 你需要设置cpu infer数量和自己单个numa上的核数相当 > > /workspace# lscpu | grep numa -i NUMA node(s): 2 NUMA node0 CPU(s): 0-87 NUMA node1 CPU(s): 88-175 > > 感谢大佬，我查看了一下配置信息如上所示，那请问是需要将cpu infer数量设置成87还是175，我现在设置的84，略小于一个numa node的核心数 87合适，你编译的时候没有export USE_NUMA=1就行。另外，max_response_tokens这个参数应该是无效的

bsakaguchi commented 2026-10-29 14:41:58 +00:00

Author

Owner

Copy link

@lessmore991 commented on GitHub (Mar 11, 2025):

你需要设置cpu infer数量和自己单个numa上的核数相当

/workspace# lscpu | grep numa -i NUMA node(s): 2 NUMA node0 CPU(s): 0-87 NUMA node1 CPU(s): 88-175
感谢大佬，我查看了一下配置信息如上所示，那请问是需要将cpu infer数量设置成87还是175，我现在设置的84，略小于一个numa node的核心数

87合适，你编译的时候没有export USE_NUMA=1就行。另外，max_response_tokens这个参数应该是无效的

enen好的，感谢指导，我也把max new tokens参数调小点试试

<!-- gh-comment-id:2713924222 --> @lessmore991 commented on GitHub (Mar 11, 2025): > > > 你需要设置cpu infer数量和自己单个numa上的核数相当 > > > > > > /workspace# lscpu | grep numa -i NUMA node(s): 2 NUMA node0 CPU(s): 0-87 NUMA node1 CPU(s): 88-175 > > 感谢大佬，我查看了一下配置信息如上所示，那请问是需要将cpu infer数量设置成87还是175，我现在设置的84，略小于一个numa node的核心数 > > 87合适，你编译的时候没有export USE_NUMA=1就行。另外，max_response_tokens这个参数应该是无效的 enen好的，感谢指导，我也把max new tokens参数调小点试试

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

auto/sync-sglang

gh-pages

docs/kt-ft-figure2-single-column-20261010

perf/bf16-grad-norm-20261010

docs/kt-ft-paper-20261008

release/20260911-carrier-repair

release/20260911-ci-python

release/20260911-pypi

ci/four-main-pypi-release

ci/model-e2e-acceptance

release/0.7.0.post3-frozen-pypi

fix/glm53-index-share-sync

release/0.7.0.post2-frozen-pypi

hotfix/0.7.0.post2-pypi-runtime

fix/avx2-wheel-feature-isolation

fix/release-self-hosted-runner

release/automate-kt-sft-pypi

support-glm5-next-djw

docs/deepseek-v4-flash-docker

ovowei-patch-3

support-glm52-djw

doc-reorg-entrypoints-support-matrix

bump-sglang-pr31

docs-v061-refresh

fp4-moe-amx

flatten-ktransformers-shim

archive-kt-sft

kt-kernel-torch-range-v061

sft-whl

sft

docs/gosim-2026-and-roadmap-link

ovowei-patch-2

ovowei-patch-1

revert-1918-fix/rocm-cpuinfer-stream-bridge

chore/kt-layerwise-prefill-main

chore/kt-layerwise-prefill-label

sft-rel

support-glm51-djw

release/v0.5.3

chore/kt-run-numa-nodes-unified

mtp

ksft-sglang

avx2

fix/sglang-kt-detection

layerwise-prefill-opt

feat/mistral-loader-support

fix-docker-build

minimax-m2.5

qwen3.5

add-maintainers

oql-doc

cli-dev-ljq

ljq-cli-dev-new

JimmyPeilinLi-patch-2

kimi_k2.5

oql-dev

experts_sched

JimmyPeilinLi-patch-1

ErvinXie-patch-1

ksft-llama

glm47_fp8

debug_llama

refactor_sft

ljq-cli-dev

refactor_sft_int4_debug

lut-nvfp4

fix-import

fix-deploy-djw

cpu-fix

ljq-dev-1

route-moe

sosp25-ae

v0.7.2

paper-ktft-v1.1

paper-ktft-v1

v0.7.1

v0.7.0.post4

v0.7.0.post1

v0.7.0

v0.6.4

v0.6.3.post1

v0.6.3

v0.6.2.post3

v0.6.2.post2

v0.6.2.post1

v0.6.2

v0.6.1

v0.6.1.post1

v0.5.3

v0.5.2.post1

v0.5.1

v0.5.0.post1

v0.5.0

v0.4.4

v0.4.3

v0.4.2

v0.4.1

v0.3.2

v0.3.1

v0.3

v0.2.4post1

v0.2.4

v0.2.3post2

v0.2.3post1

v0.2.3

v0.2.2

v0.2.2rc2

v0.2.2rc1

v0.2.1.post1

v0.2.1

v0.2.0

v0.1.4

v0.1.3

v0.1.2

v0.1.1

v0.1.0

Labels

Clear labels   

bug

  

duplicate

  

enhancement

  

good first issue

  

help wanted

  

pull-request

Mirrored from GitHub Pull Request

  

stale

No labels

bug

duplicate

enhancement

good first issue

help wanted

pull-request

stale

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/ktransformers#611

No description provided.