[GH-ISSUE #1511] [Bug] Missing required parameters of construction of `StaticCache` #1010

New issue

Open

opened 2026-10-29 14:47:27 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 14:47:27 +00:00

Owner

Originally created by @YiD11 on GitHub (Sep 20, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1511

检查清单

1. 我已经搜索过相关问题，但未能获得预期的帮助
2. 该问题在最新版本中尚未修复
3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性
4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭

问题描述 Problem Description

首轮对话的在输入后至回答前，报错：
Error reported from input to response in the first round of dialogue:

Exception has occurred: TypeError
Cache.__init__() missing 1 required positional argument: 'layer_classes'
  File "ktransformers/models/custom_cache.py", line 38, in __init__
    Cache.__init__(self)
    ~~~~~~~~~~~~~~^^^^^^
  File "ktransformers/util/utils.py", line 308, in prefill_and_generate
        config = model.config, max_batch_size = 1, max_cache_len = seq_length + max_new_tokens, device = device_map, dtype = model.dtype
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    
    
    )
    ^
  File "ktransformers/local_chat.py", line 191, in local_chat
        model, tokenizer, input_tensor.to(device), max_new_tokens, use_cuda_graph, mode = mode, force_think = force_think, chunk_size = chunk_size,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    
    
    )
    ^
  File "ktransformers/local_chat.py", line 197, in <module>
    fire.Fire(local_chat)
    ~~~~~~~~~^^^^^^^^^^^^
TypeError: Cache.__init__() missing 1 required positional argument: 'layer_classes'

复现步骤

ktransformers/local_chat.py --model_path deepseek-ai/DeepSeek-V2-Lite-Chat --gguf_path my/path/to/DeepSeek-V2-Lite-Chat-GGUF

环境信息

pip list

accelerate                1.10.1
aiohappyeyeballs          2.6.1
aiohttp                   3.12.15
aiosignal                 1.4.0
annotated-types           0.7.0
anyio                     4.9.0
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asttokens                 3.0.0
async-lru                 2.0.5
attrs                     25.3.0
babel                     2.17.0
beautifulsoup4            4.13.4
bleach                    6.2.0
blessed                   1.22.0
blobfile                  3.1.0
build                     1.3.0
certifi                   2025.4.26
cffi                      1.17.1
charset-normalizer        3.4.2
click                     8.3.0
colorlog                  6.9.0
comm                      0.2.2
contourpy                 1.3.3
cpufeature                0.2.1
cycler                    0.12.1
datasets                  4.1.1
debugpy                   1.8.14
decorator                 5.2.1
defusedxml                0.7.1
dill                      0.4.0
distro                    1.9.0
executing                 2.2.0
fastapi                   0.116.2
fastjsonschema            2.21.1
filelock                  3.19.1
fire                      0.7.1
flashinfer-python         0.2.3
fonttools                 4.60.0
fqdn                      1.5.1
frozenlist                1.7.0
fsspec                    2025.9.0
greenlet                  3.2.4
h11                       0.16.0
hf-xet                    1.1.10
httpcore                  1.0.9
httpx                     0.28.1
huggingface-hub           0.35.0
idna                      3.10
ipykernel                 6.30.1
ipython                   9.2.0
ipython_pygments_lexers   1.1.1
ipywidgets                8.1.7
isoduration               20.11.0
jedi                      0.19.2
Jinja2                    3.1.6
jiter                     0.11.0
joblib                    1.5.2
json5                     0.12.0
jsonpatch                 1.33
jsonpointer               3.0.0
jsonschema                4.23.0
jsonschema-specifications 2025.4.1
jupyter                   1.1.1
jupyter_client            8.6.3
jupyter-console           6.6.3
jupyter_core              5.7.2
jupyter-events            0.12.0
jupyter-lsp               2.2.5
jupyter_server            2.15.0
jupyter_server_terminals  0.5.3
jupyterlab                4.4.7
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.3
jupyterlab_widgets        3.0.15
kiwisolver                1.4.9
ktransformers             0.3.2+cu128torch28fancy /home/zhangyf/ktransformers
langchain                 0.3.27
langchain-core            0.3.76
langchain-text-splitters  0.3.11
langsmith                 0.4.29
lxml                      6.0.1
MarkupSafe                3.0.2
matplotlib                3.10.6
matplotlib-inline         0.1.7
mistune                   3.1.3
mpmath                    1.3.0
multidict                 6.6.4
multiprocess              0.70.16
nbclient                  0.10.2
nbconvert                 7.16.6
nbformat                  5.10.4
nest-asyncio              1.6.0
networkx                  3.5
ninja                     1.13.0
notebook                  7.4.2
notebook_shim             0.2.4
numpy                     2.3.3
nvidia-cublas-cu12        12.8.4.1
nvidia-cuda-cupti-cu12    12.8.90
nvidia-cuda-nvrtc-cu12    12.8.93
nvidia-cuda-runtime-cu12  12.8.90
nvidia-cudnn-cu12         9.10.2.21
nvidia-cufft-cu12         11.3.3.83
nvidia-cufile-cu12        1.13.1.3
nvidia-curand-cu12        10.3.9.90
nvidia-cusolver-cu12      11.7.3.90
nvidia-cusparse-cu12      12.5.8.93
nvidia-cusparselt-cu12    0.7.1
nvidia-nccl-cu12          2.27.3
nvidia-nvjitlink-cu12     12.8.93
nvidia-nvtx-cu12          12.8.90
openai                    1.108.1
orjson                    3.11.3
overrides                 7.7.0
packaging                 25.0
pandas                    2.3.2
pandocfilters             1.5.1
parso                     0.8.4
pexpect                   4.9.0
pillow                    11.3.0
pip                       25.1
platformdirs              4.3.7
prometheus_client         0.21.1
prompt_toolkit            3.0.51
propcache                 0.3.2
protobuf                  6.32.1
psutil                    7.0.0
ptyprocess                0.7.0
pure_eval                 0.2.3
pyarrow                   21.0.0
pybind11                  3.0.1
pycparser                 2.22
pycryptodomex             3.23.0
pydantic                  2.11.9
pydantic_core             2.33.2
Pygments                  2.19.1
pyparsing                 3.2.4
pyproject_hooks           1.2.0
python-dateutil           2.9.0.post0
python-json-logger        3.3.0
pytz                      2025.2
PyYAML                    6.0.2
pyzmq                     26.4.0
referencing               0.36.2
regex                     2025.9.18
requests                  2.32.3
requests-toolbelt         1.0.0
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rpds-py                   0.24.0
safetensors               0.6.2
scikit-learn              1.7.2
scipy                     1.16.2
Send2Trash                1.8.3
sentencepiece             0.2.1
setuptools                78.1.1
six                       1.17.0
sniffio                   1.3.1
soupsieve                 2.7
SQLAlchemy                2.0.43
stack-data                0.6.3
starlette                 0.48.0
sympy                     1.14.0
tenacity                  9.1.2
termcolor                 3.1.0
terminado                 0.18.1
threadpoolctl             3.6.0
tiktoken                  0.11.0
tinycss2                  1.4.0
tokenizers                0.21.4
torch                     2.8.0
tornado                   6.4.2
tqdm                      4.67.1
traitlets                 5.14.3
transformers              4.54.0
triton                    3.4.0
types-python-dateutil     2.9.0.20241206
typing_extensions         4.13.2
typing-inspection         0.4.1
tzdata                    2025.2
uri-template              1.3.0
urllib3                   2.4.0
uvicorn                   0.36.0
wcwidth                   0.2.13
webcolors                 24.11.1
webencodings              0.5.1
websocket-client          1.8.0
wheel                     0.45.1
widgetsnbextension        4.0.14
xxhash                    3.5.0
yarl                      1.20.1
zmq                       0.0.0
zstandard                 0.25.0

GPU: V100
CUDA Version: 12.9

Originally created by @YiD11 on GitHub (Sep 20, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1511 ### 检查清单 - [x] 1. 我已经搜索过相关问题，但未能获得预期的帮助 - [x] 2. 该问题在最新版本中尚未修复 - [x] 3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性 - [x] 4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭 - [x] 5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭 ### 问题描述 Problem Description 首轮对话的在输入后至回答前，报错： Error reported from input to response in the first round of dialogue: ``` text Exception has occurred: TypeError Cache.__init__() missing 1 required positional argument: 'layer_classes' File "ktransformers/models/custom_cache.py", line 38, in __init__ Cache.__init__(self) ~~~~~~~~~~~~~~^^^^^^ File "ktransformers/util/utils.py", line 308, in prefill_and_generate config = model.config, max_batch_size = 1, max_cache_len = seq_length + max_new_tokens, device = device_map, dtype = model.dtype ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "ktransformers/local_chat.py", line 191, in local_chat model, tokenizer, input_tensor.to(device), max_new_tokens, use_cuda_graph, mode = mode, force_think = force_think, chunk_size = chunk_size, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "ktransformers/local_chat.py", line 197, in <module> fire.Fire(local_chat) ~~~~~~~~~^^^^^^^^^^^^ TypeError: Cache.__init__() missing 1 required positional argument: 'layer_classes' ``` ### 复现步骤 ``` text ktransformers/local_chat.py --model_path deepseek-ai/DeepSeek-V2-Lite-Chat --gguf_path my/path/to/DeepSeek-V2-Lite-Chat-GGUF ``` ### 环境信息 pip list ``` text accelerate 1.10.1 aiohappyeyeballs 2.6.1 aiohttp 3.12.15 aiosignal 1.4.0 annotated-types 0.7.0 anyio 4.9.0 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 3.0.0 async-lru 2.0.5 attrs 25.3.0 babel 2.17.0 beautifulsoup4 4.13.4 bleach 6.2.0 blessed 1.22.0 blobfile 3.1.0 build 1.3.0 certifi 2025.4.26 cffi 1.17.1 charset-normalizer 3.4.2 click 8.3.0 colorlog 6.9.0 comm 0.2.2 contourpy 1.3.3 cpufeature 0.2.1 cycler 0.12.1 datasets 4.1.1 debugpy 1.8.14 decorator 5.2.1 defusedxml 0.7.1 dill 0.4.0 distro 1.9.0 executing 2.2.0 fastapi 0.116.2 fastjsonschema 2.21.1 filelock 3.19.1 fire 0.7.1 flashinfer-python 0.2.3 fonttools 4.60.0 fqdn 1.5.1 frozenlist 1.7.0 fsspec 2025.9.0 greenlet 3.2.4 h11 0.16.0 hf-xet 1.1.10 httpcore 1.0.9 httpx 0.28.1 huggingface-hub 0.35.0 idna 3.10 ipykernel 6.30.1 ipython 9.2.0 ipython_pygments_lexers 1.1.1 ipywidgets 8.1.7 isoduration 20.11.0 jedi 0.19.2 Jinja2 3.1.6 jiter 0.11.0 joblib 1.5.2 json5 0.12.0 jsonpatch 1.33 jsonpointer 3.0.0 jsonschema 4.23.0 jsonschema-specifications 2025.4.1 jupyter 1.1.1 jupyter_client 8.6.3 jupyter-console 6.6.3 jupyter_core 5.7.2 jupyter-events 0.12.0 jupyter-lsp 2.2.5 jupyter_server 2.15.0 jupyter_server_terminals 0.5.3 jupyterlab 4.4.7 jupyterlab_pygments 0.3.0 jupyterlab_server 2.27.3 jupyterlab_widgets 3.0.15 kiwisolver 1.4.9 ktransformers 0.3.2+cu128torch28fancy /home/zhangyf/ktransformers langchain 0.3.27 langchain-core 0.3.76 langchain-text-splitters 0.3.11 langsmith 0.4.29 lxml 6.0.1 MarkupSafe 3.0.2 matplotlib 3.10.6 matplotlib-inline 0.1.7 mistune 3.1.3 mpmath 1.3.0 multidict 6.6.4 multiprocess 0.70.16 nbclient 0.10.2 nbconvert 7.16.6 nbformat 5.10.4 nest-asyncio 1.6.0 networkx 3.5 ninja 1.13.0 notebook 7.4.2 notebook_shim 0.2.4 numpy 2.3.3 nvidia-cublas-cu12 12.8.4.1 nvidia-cuda-cupti-cu12 12.8.90 nvidia-cuda-nvrtc-cu12 12.8.93 nvidia-cuda-runtime-cu12 12.8.90 nvidia-cudnn-cu12 9.10.2.21 nvidia-cufft-cu12 11.3.3.83 nvidia-cufile-cu12 1.13.1.3 nvidia-curand-cu12 10.3.9.90 nvidia-cusolver-cu12 11.7.3.90 nvidia-cusparse-cu12 12.5.8.93 nvidia-cusparselt-cu12 0.7.1 nvidia-nccl-cu12 2.27.3 nvidia-nvjitlink-cu12 12.8.93 nvidia-nvtx-cu12 12.8.90 openai 1.108.1 orjson 3.11.3 overrides 7.7.0 packaging 25.0 pandas 2.3.2 pandocfilters 1.5.1 parso 0.8.4 pexpect 4.9.0 pillow 11.3.0 pip 25.1 platformdirs 4.3.7 prometheus_client 0.21.1 prompt_toolkit 3.0.51 propcache 0.3.2 protobuf 6.32.1 psutil 7.0.0 ptyprocess 0.7.0 pure_eval 0.2.3 pyarrow 21.0.0 pybind11 3.0.1 pycparser 2.22 pycryptodomex 3.23.0 pydantic 2.11.9 pydantic_core 2.33.2 Pygments 2.19.1 pyparsing 3.2.4 pyproject_hooks 1.2.0 python-dateutil 2.9.0.post0 python-json-logger 3.3.0 pytz 2025.2 PyYAML 6.0.2 pyzmq 26.4.0 referencing 0.36.2 regex 2025.9.18 requests 2.32.3 requests-toolbelt 1.0.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rpds-py 0.24.0 safetensors 0.6.2 scikit-learn 1.7.2 scipy 1.16.2 Send2Trash 1.8.3 sentencepiece 0.2.1 setuptools 78.1.1 six 1.17.0 sniffio 1.3.1 soupsieve 2.7 SQLAlchemy 2.0.43 stack-data 0.6.3 starlette 0.48.0 sympy 1.14.0 tenacity 9.1.2 termcolor 3.1.0 terminado 0.18.1 threadpoolctl 3.6.0 tiktoken 0.11.0 tinycss2 1.4.0 tokenizers 0.21.4 torch 2.8.0 tornado 6.4.2 tqdm 4.67.1 traitlets 5.14.3 transformers 4.54.0 triton 3.4.0 types-python-dateutil 2.9.0.20241206 typing_extensions 4.13.2 typing-inspection 0.4.1 tzdata 2025.2 uri-template 1.3.0 urllib3 2.4.0 uvicorn 0.36.0 wcwidth 0.2.13 webcolors 24.11.1 webencodings 0.5.1 websocket-client 1.8.0 wheel 0.45.1 widgetsnbextension 4.0.14 xxhash 3.5.0 yarl 1.20.1 zmq 0.0.0 zstandard 0.25.0 ``` GPU: V100 CUDA Version: 12.9

bsakaguchi commented

2026-10-29 14:47:27 +00:00

Author

Owner

@LouieBHLu commented on GitHub (Sep 21, 2025):

the same issue there.

@LouieBHLu commented on GitHub (Sep 21, 2025): the same issue there.

bsakaguchi commented

2026-10-29 14:47:28 +00:00

Author

Owner

@yedemon commented on GitHub (Oct 11, 2025):

the same issue here.

@yedemon commented on GitHub (Oct 11, 2025): the same issue here.

bsakaguchi commented

2026-10-29 14:47:28 +00:00

Author

Owner

@rhyswynn commented on GitHub (Dec 14, 2025):

pip install --upgrade transformers

pip install --upgrade "transformers[torch]" # if GPU enabled/using torch

I just had this exact issue and solved it today by upgrading transformers. The newer version has a different Cache class definition, and constructors for the older class give this error.

@rhyswynn commented on GitHub (Dec 14, 2025): pip install --upgrade transformers pip install --upgrade "transformers[torch]" # if GPU enabled/using torch I just had this exact issue and solved it today by upgrading transformers. The newer version has a different Cache class definition, and constructors for the older class give this error.