kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1120] [Bug] Custom config.json for Maverick #782

New issue
Open
opened 2026-10-29 14:44:25 +00:00 by bsakaguchi · 10 comments
bsakaguchi commented 2026-10-29 14:44:25 +00:00
Owner
Copy link

Originally created by @RodriMora on GitHub (Apr 11, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1120

Originally assigned to: @ovowei on GitHub.

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
  • 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

When trying to load the llama branch with the Scout model, it works. But using the Maverick model it doesn't work.
It needs a custom config.json file, there is a existing one for Scout but not for Marevick I believe:
https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/config.json

Injecting lm_head as ktransformers.operators.linear . KTransformersLinear
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/server/backend/interfaces/balance_serve.py", line 263, in run_engine
    engine = Engine(args, token_queue, broadcast_endpoint)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/server/backend/interfaces/balance_serve.py", line 178, in __init__
    optimize_and_load_gguf(self.model, optimize_config_path, gguf_path, config)
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/optimize/optimize.py", line 130, in optimize_and_load_gguf
    load_weights(module.lm_head, gguf_loader, "lm_head.")
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/util/utils.py", line 127, in load_weights
    module.load()
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/operators/linear.py", line 839, in load
    self.generate_linear.load(w=w)
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/operators/linear.py", line 451, in load
    w = self.load_weight(device=device)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/operators/linear.py", line 88, in load_weight
    tensor = self.gguf_loader.safetensor_loader.load_tensor(key+'.weight')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/util/custom_loader.py", line 61, in load_tensor
    raise KeyError(f"Key {key} not found in Safetensor files")
KeyError: 'Key output.weight not found in Safetensor files'

Reproduction

Launching with Maverick with this command fails with the above error:
--model_path has the modified config.json file provided by the docs
CUDA_VISIBLE_DEVICES=4 ktransformers --model_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct/ --gguf_path /mnt/llms/models/Llama-4-Maverick-17B-128E-Instruct-Q3_K_L.gguf --port 5001 --host 0.0.0.0 --cpu_infer 24 --cache_lens=12000 --max_new_tokens 1024 --optimize_config_path ktransformers/optimize/optimize_rules/Llama4-serve.yaml --backend_type balance_serve

Using the original Maverick repo gets stuck
CUDA_VISIBLE_DEVICES=4 ktransformers --model_path /mnt/llms/models/Llama-4-Maverick-17B-128E-Instruct/ --gguf_path /mnt/llms/models/Llama-4-Maverick-17B-128E-Instruct-Q3_K_L.gguf --port 5001 --host 0.0.0.0 --cpu_infer 24 --cache_lens=12000 --max_new_tokens 1024 --optimize_config_path ktransformers/optimize/optimize_rules/Llama4-serve.yaml --backend_type balance_serve

Environment

python3.11

python -m pip list:

accelerate 1.6.0
annotated-types 0.7.0
anyio 4.9.0
blessed 1.20.0
blobfile 3.0.0
build 1.2.2.post1
certifi 2025.1.31
charset-normalizer 3.4.1
click 8.1.8
colorlog 6.9.0
cpufeature 0.2.1
distro 1.9.0
fastapi 0.115.12
filelock 3.13.1
fire 0.7.0
flashinfer-python 0.2.3
fsspec 2024.6.1
greenlet 3.1.1
h11 0.14.0
httpcore 1.0.7
httpx 0.28.1
huggingface-hub 0.30.2
idna 3.10
Jinja2 3.1.4
jiter 0.9.0
jsonpatch 1.33
jsonpointer 3.0.0
ktransformers 0.2.4.post1+cu128torch26avx2
langchain 0.3.23
langchain-core 0.3.51
langchain-text-splitters 0.3.8
langsmith 0.3.30
lxml 5.3.2
MarkupSafe 2.1.5
mpmath 1.3.0
networkx 3.3
ninja 1.11.1.4
numpy 2.2.4
nvidia-cublas-cu12 12.6.4.1
nvidia-cuda-cupti-cu12 12.6.80
nvidia-cuda-nvrtc-cu12 12.6.77
nvidia-cuda-runtime-cu12 12.6.77
nvidia-cudnn-cu12 9.5.1.17
nvidia-cufft-cu12 11.3.0.4
nvidia-curand-cu12 10.3.7.77
nvidia-cusolver-cu12 11.7.1.2
nvidia-cusparse-cu12 12.5.4.2
nvidia-cusparselt-cu12 0.6.3
nvidia-nccl-cu12 2.21.5
nvidia-nvjitlink-cu12 12.6.85
nvidia-nvtx-cu12 12.6.77
openai 1.72.0
orjson 3.10.16
packaging 24.2
pillow 11.0.0
pip 24.0
protobuf 6.30.2
psutil 7.0.0
pycryptodomex 3.22.0
pydantic 2.11.3
pydantic_core 2.33.1
pyproject_hooks 1.2.0
PyYAML 6.0.2
pyzmq 26.4.0
regex 2024.11.6
requests 2.32.3
requests-toolbelt 1.0.0
safetensors 0.5.3
sentencepiece 0.2.0
setuptools 65.5.0
six 1.17.0
sniffio 1.3.1
SQLAlchemy 2.0.40
starlette 0.46.1
sympy 1.13.1
tenacity 9.1.2
termcolor 3.0.1
tiktoken 0.9.0
tokenizers 0.21.1
torch 2.6.0+cu126
torchaudio 2.6.0+cu126
torchvision 0.21.0+cu126
tqdm 4.67.1
transformers 4.51.0
triton 3.2.0
typing_extensions 4.13.2
typing-inspection 0.4.0
urllib3 2.4.0
uvicorn 0.34.0
wcwidth 0.2.13
wheel 0.45.1
zmq 0.0.0
zstandard 0.23.0

NVIDIA-SMI 570.124.06

CUDA Version: 12.8

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0
nvidia-smi -L

GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 5090
GPU 2: NVIDIA GeForce RTX 3090
GPU 3: NVIDIA GeForce RTX 3090
GPU 4: NVIDIA GeForce RTX 3090

Originally created by @RodriMora on GitHub (Apr 11, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1120 Originally assigned to: @ovowei on GitHub. ### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. - [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback. - [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. ### Describe the bug When trying to load the llama branch with the Scout model, it works. But using the Maverick model it doesn't work. It needs a custom config.json file, there is a existing one for Scout but not for Marevick I believe: https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/config.json ``` Injecting lm_head as ktransformers.operators.linear . KTransformersLinear Process SpawnProcess-1: Traceback (most recent call last): File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/server/backend/interfaces/balance_serve.py", line 263, in run_engine engine = Engine(args, token_queue, broadcast_endpoint) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/server/backend/interfaces/balance_serve.py", line 178, in __init__ optimize_and_load_gguf(self.model, optimize_config_path, gguf_path, config) File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/optimize/optimize.py", line 130, in optimize_and_load_gguf load_weights(module.lm_head, gguf_loader, "lm_head.") File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/util/utils.py", line 127, in load_weights module.load() File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/operators/linear.py", line 839, in load self.generate_linear.load(w=w) File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/operators/linear.py", line 451, in load w = self.load_weight(device=device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/operators/linear.py", line 88, in load_weight tensor = self.gguf_loader.safetensor_loader.load_tensor(key+'.weight') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/balance_serve/../../../ktransformers/util/custom_loader.py", line 61, in load_tensor raise KeyError(f"Key {key} not found in Safetensor files") KeyError: 'Key output.weight not found in Safetensor files' ``` ### Reproduction Launching with Maverick with this command fails with the above error: --model_path has the modified config.json file provided by the docs `CUDA_VISIBLE_DEVICES=4 ktransformers --model_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct/ --gguf_path /mnt/llms/models/Llama-4-Maverick-17B-128E-Instruct-Q3_K_L.gguf --port 5001 --host 0.0.0.0 --cpu_infer 24 --cache_lens=12000 --max_new_tokens 1024 --optimize_config_path ktransformers/optimize/optimize_rules/Llama4-serve.yaml --backend_type balance_serve` Using the original Maverick repo gets stuck `CUDA_VISIBLE_DEVICES=4 ktransformers --model_path /mnt/llms/models/Llama-4-Maverick-17B-128E-Instruct/ --gguf_path /mnt/llms/models/Llama-4-Maverick-17B-128E-Instruct-Q3_K_L.gguf --port 5001 --host 0.0.0.0 --cpu_infer 24 --cache_lens=12000 --max_new_tokens 1024 --optimize_config_path ktransformers/optimize/optimize_rules/Llama4-serve.yaml --backend_type balance_serve` ### Environment python3.11 python -m pip list: accelerate 1.6.0 annotated-types 0.7.0 anyio 4.9.0 blessed 1.20.0 blobfile 3.0.0 build 1.2.2.post1 certifi 2025.1.31 charset-normalizer 3.4.1 click 8.1.8 colorlog 6.9.0 cpufeature 0.2.1 distro 1.9.0 fastapi 0.115.12 filelock 3.13.1 fire 0.7.0 flashinfer-python 0.2.3 fsspec 2024.6.1 greenlet 3.1.1 h11 0.14.0 httpcore 1.0.7 httpx 0.28.1 huggingface-hub 0.30.2 idna 3.10 Jinja2 3.1.4 jiter 0.9.0 jsonpatch 1.33 jsonpointer 3.0.0 ktransformers 0.2.4.post1+cu128torch26avx2 langchain 0.3.23 langchain-core 0.3.51 langchain-text-splitters 0.3.8 langsmith 0.3.30 lxml 5.3.2 MarkupSafe 2.1.5 mpmath 1.3.0 networkx 3.3 ninja 1.11.1.4 numpy 2.2.4 nvidia-cublas-cu12 12.6.4.1 nvidia-cuda-cupti-cu12 12.6.80 nvidia-cuda-nvrtc-cu12 12.6.77 nvidia-cuda-runtime-cu12 12.6.77 nvidia-cudnn-cu12 9.5.1.17 nvidia-cufft-cu12 11.3.0.4 nvidia-curand-cu12 10.3.7.77 nvidia-cusolver-cu12 11.7.1.2 nvidia-cusparse-cu12 12.5.4.2 nvidia-cusparselt-cu12 0.6.3 nvidia-nccl-cu12 2.21.5 nvidia-nvjitlink-cu12 12.6.85 nvidia-nvtx-cu12 12.6.77 openai 1.72.0 orjson 3.10.16 packaging 24.2 pillow 11.0.0 pip 24.0 protobuf 6.30.2 psutil 7.0.0 pycryptodomex 3.22.0 pydantic 2.11.3 pydantic_core 2.33.1 pyproject_hooks 1.2.0 PyYAML 6.0.2 pyzmq 26.4.0 regex 2024.11.6 requests 2.32.3 requests-toolbelt 1.0.0 safetensors 0.5.3 sentencepiece 0.2.0 setuptools 65.5.0 six 1.17.0 sniffio 1.3.1 SQLAlchemy 2.0.40 starlette 0.46.1 sympy 1.13.1 tenacity 9.1.2 termcolor 3.0.1 tiktoken 0.9.0 tokenizers 0.21.1 torch 2.6.0+cu126 torchaudio 2.6.0+cu126 torchvision 0.21.0+cu126 tqdm 4.67.1 transformers 4.51.0 triton 3.2.0 typing_extensions 4.13.2 typing-inspection 0.4.0 urllib3 2.4.0 uvicorn 0.34.0 wcwidth 0.2.13 wheel 0.45.1 zmq 0.0.0 zstandard 0.23.0 NVIDIA-SMI 570.124.06 CUDA Version: 12.8 nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Fri_Feb_21_20:23:50_PST_2025 Cuda compilation tools, release 12.8, V12.8.93 Build cuda_12.8.r12.8/compiler.35583870_0 nvidia-smi -L GPU 0: NVIDIA GeForce RTX 3090 GPU 1: NVIDIA GeForce RTX 5090 GPU 2: NVIDIA GeForce RTX 3090 GPU 3: NVIDIA GeForce RTX 3090 GPU 4: NVIDIA GeForce RTX 3090
bsakaguchi commented 2026-10-29 14:44:25 +00:00
Author
Owner
Copy link

@git-shan commented on GitHub (Apr 11, 2025):

Got same error when loading Marveric. one more error message below:
ValueError: Experts blk.0 not found in gguf_loader

<!-- gh-comment-id:2796869735 --> @git-shan commented on GitHub (Apr 11, 2025): Got same error when loading Marveric. one more error message below: ValueError: Experts blk.0 not found in gguf_loader
bsakaguchi commented 2026-10-29 14:44:25 +00:00
Author
Owner
Copy link

@igerry commented on GitHub (Apr 12, 2025):

Congraturations.
got the same error on Marveric

<!-- gh-comment-id:2798378766 --> @igerry commented on GitHub (Apr 12, 2025): Congraturations. got the same error on Marveric
bsakaguchi commented 2026-10-29 14:44:26 +00:00
Author
Owner
Copy link

@ovowei commented on GitHub (Apr 15, 2025):

Thank you for your report. This issue is caused by structural differences between the Maverick and Scout models. I’ll fix it in a few days.

<!-- gh-comment-id:2804308297 --> @ovowei commented on GitHub (Apr 15, 2025): Thank you for your report. This issue is caused by structural differences between the Maverick and Scout models. I’ll fix it in a few days.
bsakaguchi commented 2026-10-29 14:44:26 +00:00
Author
Owner
Copy link

@adonishong commented on GitHub (Apr 15, 2025):

Thank you for your report. This issue is caused by structural differences between the Maverick and Scout models. I’ll fix it in a few days.

Looking forward to this ... LOL

<!-- gh-comment-id:2804676029 --> @adonishong commented on GitHub (Apr 15, 2025): > Thank you for your report. This issue is caused by structural differences between the Maverick and Scout models. I’ll fix it in a few days. Looking forward to this ... LOL
bsakaguchi commented 2026-10-29 14:44:26 +00:00
Author
Owner
Copy link

@airnsk commented on GitHub (Apr 17, 2025):

Fix bug when loading Marveric Model #1160
does not work:
raise ValueError(f"Experts {key} not found in gguf_loader")
ValueError: Experts blk.0 not found in gguf_loader

<!-- gh-comment-id:2813765221 --> @airnsk commented on GitHub (Apr 17, 2025): [Fix bug when loading Marveric Model #1160](https://github.com/kvcache-ai/ktransformers/pull/1160) does not work: raise ValueError(f"Experts {key} not found in gguf_loader") ValueError: Experts blk.0 not found in gguf_loader
bsakaguchi commented 2026-10-29 14:44:26 +00:00
Author
Owner
Copy link

@ovowei commented on GitHub (Apr 18, 2025):

Try reinstalling.

<!-- gh-comment-id:2814446853 --> @ovowei commented on GitHub (Apr 18, 2025): Try reinstalling.
bsakaguchi commented 2026-10-29 14:44:26 +00:00
Author
Owner
Copy link

@ovowei commented on GitHub (Apr 18, 2025):

I suspect this issue is caused by an incorrect config file. The custom config for the Maverick model is different from the one used for the Scout model. I've updated both of them:

<!-- gh-comment-id:2814471010 --> @ovowei commented on GitHub (Apr 18, 2025): I suspect this issue is caused by an incorrect config file. The custom config for the Maverick model is different from the one used for the Scout model. I've updated both of them: - Use [[scout_config.json](https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/scout_config.json)](https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/scout_config.json) for the Llama-4-Scout-17B-16E model - Use [[maverick_config.json](https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/maverick_config.json)](https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/maverick_config.json) for the Llama-4-Maverick-17B-128E model
bsakaguchi commented 2026-10-29 14:44:26 +00:00
Author
Owner
Copy link

@igerry commented on GitHub (Apr 18, 2025):

Great thanks @ovowei for the fix. After being pulled and re-compiled, it starts to work.

<!-- gh-comment-id:2814566181 --> @igerry commented on GitHub (Apr 18, 2025): Great thanks @ovowei for the fix. After being pulled and re-compiled, it starts to work.
bsakaguchi commented 2026-10-29 14:44:27 +00:00
Author
Owner
Copy link

@airnsk commented on GitHub (Apr 18, 2025):

Great. on 2x2680 and 512gb(numa) +1x 3090 I get 21 t/s . Thanks @ovowei

<!-- gh-comment-id:2814763096 --> @airnsk commented on GitHub (Apr 18, 2025): Great. on 2x2680 and 512gb(numa) +1x 3090 I get 21 t/s . Thanks @ovowei
bsakaguchi commented 2026-10-29 14:44:27 +00:00
Author
Owner
Copy link

@git-shan commented on GitHub (Apr 20, 2025):

Great thanks @@ovowei !! new update of support-llama4 fully works

I suspect this issue is caused by an incorrect config file. The custom config for the Maverick model is different from the one used for the Scout model. I've updated both of them:

<!-- gh-comment-id:2817050584 --> @git-shan commented on GitHub (Apr 20, 2025): Great thanks @@ovowei !! new update of support-llama4 fully works > I suspect this issue is caused by an incorrect config file. The custom config for the Maverick model is different from the one used for the Scout model. I've updated both of them: > > * Use [[scout_config.json](https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/scout_config.json)](https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/scout_config.json) for the Llama-4-Scout-17B-16E model > * Use [[maverick_config.json](https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/maverick_config.json)](https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/maverick_config.json) for the Llama-4-Maverick-17B-128E model @
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#782
No description provided.