kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1666] KTransformers+SGLang Inference Deployment fail #1072

New issue
Closed
opened 2026-10-29 14:48:13 +00:00 by bsakaguchi · 2 comments
bsakaguchi commented 2026-10-29 14:48:13 +00:00
Owner
Copy link

Originally created by @Yunboxun on GitHub (Dec 4, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1666

Originally assigned to: @SkqLiao on GitHub.

Reminder

  • I have read the above rules and searched the existing issues.

System Info

CPU信息 (CPU Info)
架构：x86_64 (64-bit, Little Endian)
型号：Intel(R) Xeon(R) Gold 5418Y
核心配置：
物理CPU数量：2颗
每颗核心数：24核
每核线程数：2线程
总逻辑CPU数：96核
AVX-512、AMX-BF16、AMX-TILE、AMX-INT8 指令集支持

cpu内存:960G

GPU信息
GPU型号：8 × NVIDIA L20
驱动版本：570.124.06
CUDA版本：12.9
单卡规格：
显存：46,068 MiB (约45GB)

Reproduction

运行指令，是我哪里配置不对吗，--kt-threadpool-count从2-50我都试过
python3 -m sglang.launch_server --host 0.0.0.0 --port 8080 --model /hbox2dir/Kimi-K2-Thinking --kt-weight-path /hbox2dir/Kimi-K2-Thinking-CPU-weight --kt-cpuinfer 88 --kt-threadpool-count 2 --kt-num-gpu-experts 1 --kt-method AMXINT4 --attention-backend flashinfer --trust-remote-code --mem-fraction-static 0.98 --chunked-prefill-size 2048 --max-running-requests 8 --max-total-tokens 12800 --enable-mixed-chunk --tensor-parallel-size 8 --enable-p2p-check --disable-shared-experts-fusion

[2025-12-04 07:44:24 TP4] Scheduler hit an exception: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__
    self.tp_worker = TpModelWorker(
                     ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__
    self.initialize(min_per_gpu_memory)
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize
    self.load_model()
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model
    raise ValueError(
ValueError: TP rank 4 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node.

[2025-12-04 07:44:24] Received sigquit from a child process. It usually means the child failed.
[2025-12-04 07:44:24 TP7] Scheduler hit an exception: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__
    self.tp_worker = TpModelWorker(
                     ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__
    self.initialize(min_per_gpu_memory)
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize
    self.load_model()
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model
    raise ValueError(
ValueError: TP rank 7 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node.

[2025-12-04 07:44:24 TP3] Scheduler hit an exception: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__
    self.tp_worker = TpModelWorker(
                     ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__
    self.initialize(min_per_gpu_memory)
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize
    self.load_model()
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model
    raise ValueError(
ValueError: TP rank 3 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node.

[2025-12-04 07:44:24 TP1] Scheduler hit an exception: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__
    self.tp_worker = TpModelWorker(
                     ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__
    self.initialize(min_per_gpu_memory)
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize
    self.load_model()
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model
    raise ValueError(
ValueError: TP rank 1 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node.

[2025-12-04 07:44:24 TP2] Scheduler hit an exception: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__
    self.tp_worker = TpModelWorker(
                     ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__
    self.initialize(min_per_gpu_memory)
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize
    self.load_model()
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model
    raise ValueError(
ValueError: TP rank 2 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node.

[2025-12-04 07:44:24 TP5] Scheduler hit an exception: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__
    self.tp_worker = TpModelWorker(
                     ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__
    self.initialize(min_per_gpu_memory)
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize
    self.load_model()
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model
    raise ValueError(
ValueError: TP rank 5 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node.

[2025-12-04 07:44:24] Received sigquit from a child process. It usually means the child failed.
[2025-12-04 07:44:24 TP6] Scheduler hit an exception: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__
    self.tp_worker = TpModelWorker(
                     ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__
    self.initialize(min_per_gpu_memory)
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize
    self.load_model()
  File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model
    raise ValueError(
ValueError: TP rank 6 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node.

[2025-12-04 07:44:24] Received sigquit from a child process. It usually means the child failed.
Killed

Others

No response

Originally created by @Yunboxun on GitHub (Dec 4, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1666 Originally assigned to: @SkqLiao on GitHub. ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info CPU信息 (CPU Info) 架构：x86_64 (64-bit, Little Endian) 型号：Intel(R) Xeon(R) Gold 5418Y 核心配置： 物理CPU数量：2颗 每颗核心数：24核 每核线程数：2线程 总逻辑CPU数：96核 AVX-512、AMX-BF16、AMX-TILE、AMX-INT8 指令集支持 cpu内存:960G GPU信息 GPU型号：8 × NVIDIA L20 驱动版本：570.124.06 CUDA版本：12.9 单卡规格： 显存：46,068 MiB (约45GB) ### Reproduction 运行指令，是我哪里配置不对吗，--kt-threadpool-count从2-50我都试过 `python3 -m sglang.launch_server --host 0.0.0.0 --port 8080 --model /hbox2dir/Kimi-K2-Thinking --kt-weight-path /hbox2dir/Kimi-K2-Thinking-CPU-weight --kt-cpuinfer 88 --kt-threadpool-count 2 --kt-num-gpu-experts 1 --kt-method AMXINT4 --attention-backend flashinfer --trust-remote-code --mem-fraction-static 0.98 --chunked-prefill-size 2048 --max-running-requests 8 --max-total-tokens 12800 --enable-mixed-chunk --tensor-parallel-size 8 --enable-p2p-check --disable-shared-experts-fusion` ``` [2025-12-04 07:44:24 TP4] Scheduler hit an exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process scheduler = Scheduler( ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__ self.tp_worker = TpModelWorker( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__ self._model_runner = ModelRunner( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__ self.initialize(min_per_gpu_memory) File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize self.load_model() File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model raise ValueError( ValueError: TP rank 4 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node. [2025-12-04 07:44:24] Received sigquit from a child process. It usually means the child failed. [2025-12-04 07:44:24 TP7] Scheduler hit an exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process scheduler = Scheduler( ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__ self.tp_worker = TpModelWorker( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__ self._model_runner = ModelRunner( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__ self.initialize(min_per_gpu_memory) File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize self.load_model() File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model raise ValueError( ValueError: TP rank 7 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node. [2025-12-04 07:44:24 TP3] Scheduler hit an exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process scheduler = Scheduler( ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__ self.tp_worker = TpModelWorker( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__ self._model_runner = ModelRunner( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__ self.initialize(min_per_gpu_memory) File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize self.load_model() File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model raise ValueError( ValueError: TP rank 3 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node. [2025-12-04 07:44:24 TP1] Scheduler hit an exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process scheduler = Scheduler( ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__ self.tp_worker = TpModelWorker( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__ self._model_runner = ModelRunner( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__ self.initialize(min_per_gpu_memory) File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize self.load_model() File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model raise ValueError( ValueError: TP rank 1 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node. [2025-12-04 07:44:24 TP2] Scheduler hit an exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process scheduler = Scheduler( ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__ self.tp_worker = TpModelWorker( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__ self._model_runner = ModelRunner( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__ self.initialize(min_per_gpu_memory) File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize self.load_model() File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model raise ValueError( ValueError: TP rank 2 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node. [2025-12-04 07:44:24 TP5] Scheduler hit an exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process scheduler = Scheduler( ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__ self.tp_worker = TpModelWorker( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__ self._model_runner = ModelRunner( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__ self.initialize(min_per_gpu_memory) File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize self.load_model() File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model raise ValueError( ValueError: TP rank 5 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node. [2025-12-04 07:44:24] Received sigquit from a child process. It usually means the child failed. [2025-12-04 07:44:24 TP6] Scheduler hit an exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 2712, in run_scheduler_process scheduler = Scheduler( ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/scheduler.py", line 312, in __init__ self.tp_worker = TpModelWorker( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/managers/tp_worker.py", line 237, in __init__ self._model_runner = ModelRunner( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 324, in __init__ self.initialize(min_per_gpu_memory) File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 410, in initialize self.load_model() File "/usr/local/lib/python3.12/dist-packages/sglang/srt/model_executor/model_runner.py", line 849, in load_model raise ValueError( ValueError: TP rank 6 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node. [2025-12-04 07:44:24] Received sigquit from a child process. It usually means the child failed. Killed ``` ### Others _No response_
bsakaguchi 2026-10-29 14:48:13 +00:00
  • closed this issue
  • added the
    bug
         label
bsakaguchi commented 2026-10-29 14:48:15 +00:00
Author
Owner
Copy link

@SkqLiao commented on GitHub (Dec 5, 2025):

Oh, it seems to be an issue with SGLang...
Anyway, we will be releasing a new version that support native Kimi-K2INT4 support with GPU layerwise prefill, so stay tuned.

<!-- gh-comment-id:3615631073 --> @SkqLiao commented on GitHub (Dec 5, 2025): Oh, it seems to be an issue with SGLang... Anyway, we will be releasing a new version that support native Kimi-K2INT4 support with GPU layerwise prefill, so stay tuned.
bsakaguchi commented 2026-10-29 14:48:15 +00:00
Author
Owner
Copy link

@ErvinXie commented on GitHub (Dec 8, 2025):

We have already release the new version, please try new v0.4.3.

<!-- gh-comment-id:3624191950 --> @ErvinXie commented on GitHub (Dec 8, 2025): We have already release the new version, please try new v0.4.3.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#1072
No description provided.