kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1113] [Bug] Exception: data did not match any variant of untagged enum ModelWrapper #777

New issue
Closed
opened 2026-10-29 14:44:19 +00:00 by bsakaguchi · 5 comments
bsakaguchi commented 2026-10-29 14:44:19 +00:00
Owner
Copy link

Originally created by @RodriMora on GitHub (Apr 10, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1113

Originally assigned to: @ovowei on GitHub.

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
  • 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

Running the llama4 branch, it fails when loading the model.

Something I noticed is that the docs say it needs:
transformers==4.51.0

But the ktransformers/server/requirements.txt on the support-llama4 branch, requiere:
transformers == 4.43.2

Error:

(.venv) ubuntuai@ubuntuai ~/k/ktransformers (support-llama4)> CUDA_VISIBLE_DEVICES=4 ktransformers --model_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct/ --gguf_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct-UD-Q2_K_XL --port 5001 --host 0.0.0.0 --cpu_infer 24  --cache_lens=32000 --max_new_tokens 1024 --optimize_config_path ktransformers/optimize/optimize_rules/Llama4-serve.yaml --backend_type balance_serve
2025-04-10 18:43:02,411 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
sched_rpc started with PID: 1227658
flash_attn not found, flashinfer unit test needed it. If you are using balance serve, ignore this.
set start method
Traceback (most recent call last):
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/bin/ktransformers", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/main.py", line 132, in main
    create_interface(config=cfg, default_args=cfg)
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/utils/create_interface.py", line 30, in create_interface
    GlobalInterface.interface = BackendInterface(default_args)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 294, in __init__
    self.tokenizer = AutoTokenizer.from_pretrained(args.model_dir, trust_remote_code=True)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 896, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2291, in from_pretrained
    return cls._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2525, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 115, in __init__
    fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Exception: data did not match any variant of untagged enum ModelWrapper at line 1969534 column 3

Reproduction

git checkout support-llama4
git submodule update --init --recursive
python -m pip install packaging ninja cpufeature numpy openai
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
USE_BALANCE_SERVE=1  bash ./install.sh

I have the modified config.json file from here: https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/config.json into the /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct folder:

CUDA_VISIBLE_DEVICES=4 ktransformers --model_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct/ --gguf_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct-UD-Q2_K_XL --port 5000 --host 0.0.0.0 --cpu_infer 24 --cache_lens=32000 --max_new_tokens 1024 --optimize_config_path ktransformers/optimize/optimize_rules/Llama4-serve.yaml --backend_type balance_serve

Environment

python3.11

python -m pip list:

accelerate               1.6.0
annotated-types          0.7.0
anyio                    4.9.0
blessed                  1.20.0
blobfile                 3.0.0
build                    1.2.2.post1
certifi                  2025.1.31
charset-normalizer       3.4.1
click                    8.1.8
colorlog                 6.9.0
cpufeature               0.2.1
distro                   1.9.0
fastapi                  0.115.12
filelock                 3.13.1
fire                     0.7.0
flashinfer-python        0.2.3
fsspec                   2024.6.1
greenlet                 3.1.1
h11                      0.14.0
httpcore                 1.0.7
httpx                    0.28.1
huggingface-hub          0.30.2
idna                     3.10
Jinja2                   3.1.4
jiter                    0.9.0
jsonpatch                1.33
jsonpointer              3.0.0
ktransformers            0.2.4.post1+cu128torch26avx2
langchain                0.3.23
langchain-core           0.3.51
langchain-text-splitters 0.3.8
langsmith                0.3.27
lxml                     5.3.2
MarkupSafe               2.1.5
mpmath                   1.3.0
networkx                 3.3
ninja                    1.11.1.4
numpy                    2.2.4
nvidia-cublas-cu12       12.6.4.1
nvidia-cuda-cupti-cu12   12.6.80
nvidia-cuda-nvrtc-cu12   12.6.77
nvidia-cuda-runtime-cu12 12.6.77
nvidia-cudnn-cu12        9.5.1.17
nvidia-cufft-cu12        11.3.0.4
nvidia-curand-cu12       10.3.7.77
nvidia-cusolver-cu12     11.7.1.2
nvidia-cusparse-cu12     12.5.4.2
nvidia-cusparselt-cu12   0.6.3
nvidia-nccl-cu12         2.21.5
nvidia-nvjitlink-cu12    12.6.85
nvidia-nvtx-cu12         12.6.77
openai                   1.72.0
orjson                   3.10.16
packaging                24.2
pillow                   11.0.0
pip                      24.0
protobuf                 6.30.2
psutil                   7.0.0
pycryptodomex            3.22.0
pydantic                 2.11.3
pydantic_core            2.33.1
pyproject_hooks          1.2.0
PyYAML                   6.0.2
pyzmq                    26.4.0
regex                    2024.11.6
requests                 2.32.3
requests-toolbelt        1.0.0
safetensors              0.5.3
sentencepiece            0.2.0
setuptools               65.5.0
six                      1.17.0
sniffio                  1.3.1
SQLAlchemy               2.0.40
starlette                0.46.1
sympy                    1.13.1
tenacity                 9.1.2
termcolor                3.0.1
tiktoken                 0.9.0
tokenizers               0.19.1
torch                    2.6.0+cu126
torchaudio               2.6.0+cu126
torchvision              0.21.0+cu126
tqdm                     4.67.1
transformers             4.43.2
triton                   3.2.0
typing_extensions        4.13.1
typing-inspection        0.4.0
urllib3                  2.3.0
uvicorn                  0.34.0
wcwidth                  0.2.13
wheel                    0.45.1
zmq                      0.0.0
zstandard                0.23.0

NVIDIA-SMI 570.124.06

CUDA Version: 12.8

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0

nvidia-smi -L

GPU 0: NVIDIA GeForce RTX 3090 
GPU 1: NVIDIA GeForce RTX 5090 
GPU 2: NVIDIA GeForce RTX 3090
GPU 3: NVIDIA GeForce RTX 3090 
GPU 4: NVIDIA GeForce RTX 3090
Originally created by @RodriMora on GitHub (Apr 10, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1113 Originally assigned to: @ovowei on GitHub. ### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. - [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback. - [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. ### Describe the bug Running the llama4 branch, it fails when loading the model. Something I noticed is that the docs say it needs: `transformers==4.51.0` But the `ktransformers/server/requirements.txt` on the `support-llama4` branch, requiere: `transformers == 4.43.2` Error: ``` (.venv) ubuntuai@ubuntuai ~/k/ktransformers (support-llama4)> CUDA_VISIBLE_DEVICES=4 ktransformers --model_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct/ --gguf_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct-UD-Q2_K_XL --port 5001 --host 0.0.0.0 --cpu_infer 24 --cache_lens=32000 --max_new_tokens 1024 --optimize_config_path ktransformers/optimize/optimize_rules/Llama4-serve.yaml --backend_type balance_serve 2025-04-10 18:43:02,411 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend found flashinfer sched_rpc started with PID: 1227658 flash_attn not found, flashinfer unit test needed it. If you are using balance serve, ignore this. set start method Traceback (most recent call last): File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/bin/ktransformers", line 8, in <module> sys.exit(main()) ^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/main.py", line 132, in main create_interface(config=cfg, default_args=cfg) File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/utils/create_interface.py", line 30, in create_interface GlobalInterface.interface = BackendInterface(default_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 294, in __init__ self.tokenizer = AutoTokenizer.from_pretrained(args.model_dir, trust_remote_code=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 896, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2291, in from_pretrained return cls._from_pretrained( ^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2525, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntuai/ktransformers_llama4/ktransformers/.venv/lib/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 115, in __init__ fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Exception: data did not match any variant of untagged enum ModelWrapper at line 1969534 column 3 ``` ### Reproduction ``` git checkout support-llama4 git submodule update --init --recursive python -m pip install packaging ninja cpufeature numpy openai python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126 USE_BALANCE_SERVE=1 bash ./install.sh ``` I have the modified config.json file from here: https://github.com/kvcache-ai/ktransformers/blob/support-llama4/doc/en/config.json into the /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct folder: `CUDA_VISIBLE_DEVICES=4 ktransformers --model_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct/ --gguf_path /mnt/llms/models/Llama-4-Scout-17B-16E-Instruct-UD-Q2_K_XL --port 5000 --host 0.0.0.0 --cpu_infer 24 --cache_lens=32000 --max_new_tokens 1024 --optimize_config_path ktransformers/optimize/optimize_rules/Llama4-serve.yaml --backend_type balance_serve` ### Environment python3.11 python -m pip list: ``` accelerate 1.6.0 annotated-types 0.7.0 anyio 4.9.0 blessed 1.20.0 blobfile 3.0.0 build 1.2.2.post1 certifi 2025.1.31 charset-normalizer 3.4.1 click 8.1.8 colorlog 6.9.0 cpufeature 0.2.1 distro 1.9.0 fastapi 0.115.12 filelock 3.13.1 fire 0.7.0 flashinfer-python 0.2.3 fsspec 2024.6.1 greenlet 3.1.1 h11 0.14.0 httpcore 1.0.7 httpx 0.28.1 huggingface-hub 0.30.2 idna 3.10 Jinja2 3.1.4 jiter 0.9.0 jsonpatch 1.33 jsonpointer 3.0.0 ktransformers 0.2.4.post1+cu128torch26avx2 langchain 0.3.23 langchain-core 0.3.51 langchain-text-splitters 0.3.8 langsmith 0.3.27 lxml 5.3.2 MarkupSafe 2.1.5 mpmath 1.3.0 networkx 3.3 ninja 1.11.1.4 numpy 2.2.4 nvidia-cublas-cu12 12.6.4.1 nvidia-cuda-cupti-cu12 12.6.80 nvidia-cuda-nvrtc-cu12 12.6.77 nvidia-cuda-runtime-cu12 12.6.77 nvidia-cudnn-cu12 9.5.1.17 nvidia-cufft-cu12 11.3.0.4 nvidia-curand-cu12 10.3.7.77 nvidia-cusolver-cu12 11.7.1.2 nvidia-cusparse-cu12 12.5.4.2 nvidia-cusparselt-cu12 0.6.3 nvidia-nccl-cu12 2.21.5 nvidia-nvjitlink-cu12 12.6.85 nvidia-nvtx-cu12 12.6.77 openai 1.72.0 orjson 3.10.16 packaging 24.2 pillow 11.0.0 pip 24.0 protobuf 6.30.2 psutil 7.0.0 pycryptodomex 3.22.0 pydantic 2.11.3 pydantic_core 2.33.1 pyproject_hooks 1.2.0 PyYAML 6.0.2 pyzmq 26.4.0 regex 2024.11.6 requests 2.32.3 requests-toolbelt 1.0.0 safetensors 0.5.3 sentencepiece 0.2.0 setuptools 65.5.0 six 1.17.0 sniffio 1.3.1 SQLAlchemy 2.0.40 starlette 0.46.1 sympy 1.13.1 tenacity 9.1.2 termcolor 3.0.1 tiktoken 0.9.0 tokenizers 0.19.1 torch 2.6.0+cu126 torchaudio 2.6.0+cu126 torchvision 0.21.0+cu126 tqdm 4.67.1 transformers 4.43.2 triton 3.2.0 typing_extensions 4.13.1 typing-inspection 0.4.0 urllib3 2.3.0 uvicorn 0.34.0 wcwidth 0.2.13 wheel 0.45.1 zmq 0.0.0 zstandard 0.23.0 ``` NVIDIA-SMI 570.124.06 CUDA Version: 12.8 nvcc --version ``` nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Fri_Feb_21_20:23:50_PST_2025 Cuda compilation tools, release 12.8, V12.8.93 Build cuda_12.8.r12.8/compiler.35583870_0 ``` nvidia-smi -L ``` GPU 0: NVIDIA GeForce RTX 3090 GPU 1: NVIDIA GeForce RTX 5090 GPU 2: NVIDIA GeForce RTX 3090 GPU 3: NVIDIA GeForce RTX 3090 GPU 4: NVIDIA GeForce RTX 3090 ```
bsakaguchi commented 2026-10-29 14:44:20 +00:00
Author
Owner
Copy link

@KMSorSMS commented on GitHub (Apr 11, 2025):

Thx for your report. As we do need some other version of transformers, these seem like another problem that should be a bug. What's your opinion? @ovowei ？

<!-- gh-comment-id:2795679625 --> @KMSorSMS commented on GitHub (Apr 11, 2025): Thx for your report. As we do need some other version of transformers, these seem like another problem that should be a bug. What's your opinion? @ovowei ？
bsakaguchi commented 2026-10-29 14:44:20 +00:00
Author
Owner
Copy link

@ovowei commented on GitHub (Apr 11, 2025):

This is a bug — I forgot to update the requirements file under the support-llama4 branch to specify the version of transformers. It should be transformers==4.51.0. I'll fix it right away.

<!-- gh-comment-id:2795693812 --> @ovowei commented on GitHub (Apr 11, 2025): This is a bug — I forgot to update the requirements file under the support-llama4 branch to specify the version of transformers. It should be transformers==4.51.0. I'll fix it right away.
bsakaguchi commented 2026-10-29 14:44:20 +00:00
Author
Owner
Copy link

@RodriMora commented on GitHub (Apr 11, 2025):

I just installed everything from scratch with the latest commit but still got the old version:

Image

When trying to manually install version 4.51.0 i get this dependency error:

python -m pip install transformers==4.51.0

ktransformers 0.2.4.post1+cu128torch26avx2 requires transformers==4.43.2, but you have transformers 4.51.0 which is incompatible
Image

<!-- gh-comment-id:2796268017 --> @RodriMora commented on GitHub (Apr 11, 2025): I just installed everything from scratch with the latest commit but still got the old version: <img width="1309" alt="Image" src="https://github.com/user-attachments/assets/c9e2fdcf-e238-4fe8-a128-2419557252e6" /> When trying to manually install version 4.51.0 i get this dependency error: `python -m pip install transformers==4.51.0` `ktransformers 0.2.4.post1+cu128torch26avx2 requires transformers==4.43.2, but you have transformers 4.51.0 which is incompatible` <img width="1533" alt="Image" src="https://github.com/user-attachments/assets/4aef9dbb-5567-4d7b-8629-026a03c2ea0b" />
bsakaguchi commented 2026-10-29 14:44:21 +00:00
Author
Owner
Copy link

@ovowei commented on GitHub (Apr 11, 2025):

I believe I’ve already fixed this bug, maybe you haven’t pulled the latest code?
Anyway, the "incompatible" error in the terminal shouldn’t affect actual execution. You could try running it to see if it works.

<!-- gh-comment-id:2796294639 --> @ovowei commented on GitHub (Apr 11, 2025): I believe I’ve already fixed this bug, maybe you haven’t pulled the latest code? Anyway, the "incompatible" error in the terminal shouldn’t affect actual execution. You could try running it to see if it works.
bsakaguchi commented 2026-10-29 14:44:21 +00:00
Author
Owner
Copy link

@RodriMora commented on GitHub (Apr 11, 2025):

It's weird, i had the latest code (git pull)

Image

As you can see in the previous screenshot the requirements file shows the 4.51.0 version, but when building it still installed the old version, maybe python had it cached somewhere.

But as you said, installing manually even with the error it worked. Thanks!

26t/s generation

Image
<!-- gh-comment-id:2796423325 --> @RodriMora commented on GitHub (Apr 11, 2025): It's weird, i had the latest code (git pull) <img width="1256" alt="Image" src="https://github.com/user-attachments/assets/ab375ee1-84d0-4249-abf8-17440ee35abf" /> As you can see in the previous screenshot the requirements file shows the 4.51.0 version, but when building it still installed the old version, maybe python had it cached somewhere. But as you said, installing manually even with the error it worked. Thanks! 26t/s generation <img width="1541" alt="Image" src="https://github.com/user-attachments/assets/8b0e79d0-70bd-419e-b271-390e3c2e1045" />
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#777
No description provided.