[GH-ISSUE #1677] ktransformers:v0.3.1-AVX2官方镜像POST请求报错AttributeError: 'DeepseekV2ForCausalLM' object has no attribute '_get_logits_warper' #1075

New issue

Closed

opened 2026-10-29 14:48:16 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:48:16 +00:00

Owner

Originally created by @Qing-Qiu on GitHub (Dec 6, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1677

Reminder

I have read the above rules and searched the existing issues.

System Info

cpu支持avx2，不支持其它几个指令集
GPU

在windows11的wsl2中装了docker，docker拉取了ktransformers:v0.3.1-AVX2官方镜像，模型选择的是DeepSeek-V2-Lite-Chat，在modelscope下载了DeepSeek-V2-Lite-Chat-Q4_K_M.gguf作为量化权重文件。

Reproduction

进入容器后输入

python -m ktransformers.local_chat \
  --model_path /models/DeepSeek-V2-Lite-Chat \
  --gguf_path /models/DeepSeek-V2-Lite-Chat

可以正常进行对话，输入

python -m ktransformers.server.main \
  --model_path /models/DeepSeek-V2-Lite-Chat \
  --gguf_path /models/DeepSeek-V2-Lite-Chat \
  --port 11435 \
  --host 0.0.0.0

可以正常开启服务

使用chatbox或postman发送post请求

出现错误

INFO:     127.0.0.1:38502 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/workspace/ktransformers/ktransformers/server/backend/interfaces/transformers.py", line 227, in prepare_logits_wrapper
    self.model._get_logits_warper(generation_config, device=device)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1931, in __getattr__
    raise AttributeError(
AttributeError: 'DeepseekV2ForCausalLM' object has no attribute '_get_logits_warper'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/opt/conda/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/ktransformers/ktransformers/server/api/openai/endpoints/chat.py", line 436, in chat_completion
    async for res in interface.inference(input_message, id, create.temperature, create.top_p, create.max_tokens, create.max_completion_tokens):
  File "/workspace/ktransformers/ktransformers/server/backend/interfaces/ktransformers.py", line 241, in inference
    async for v in super().inference(local_messages, thread_id, temperature, top_p, max_tokens, max_completion_tokens):
  File "/workspace/ktransformers/ktransformers/server/backend/interfaces/transformers.py", line 412, in inference
    for t in self.prefill(input_ids, self.check_is_new(thread_id), temperature, top_p, max_tokens, max_completion_tokens):
  File "/opt/conda/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 36, in generator_context
    response = gen.send(None)
               ^^^^^^^^^^^^^^
  File "/workspace/ktransformers/ktransformers/server/backend/interfaces/ktransformers.py", line 229, in prefill
    self.prepare_logits_wrapper(input_ids, device, temperature, top_p)
  File "/workspace/ktransformers/ktransformers/server/backend/interfaces/transformers.py", line 231, in prepare_logits_wrapper
    self.model._get_logits_warper(generation_config)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1931, in __getattr__
    raise AttributeError(
AttributeError: 'DeepseekV2ForCausalLM' object has no attribute '_get_logits_warper'

请问这个如何解决呢？

Others

No response

Originally created by @Qing-Qiu on GitHub (Dec 6, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1677 ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info cpu支持avx2，不支持其它几个指令集 GPU <img width="828" height="216" alt="Image" src="https://github.com/user-attachments/assets/9fae37da-b29a-4050-a6f7-1d176f4104e7" /> 在windows11的wsl2中装了docker，docker拉取了ktransformers:v0.3.1-AVX2官方镜像，模型选择的是DeepSeek-V2-Lite-Chat，在modelscope下载了DeepSeek-V2-Lite-Chat-Q4_K_M.gguf作为量化权重文件。 ### Reproduction 进入容器后输入 ``` python -m ktransformers.local_chat \ --model_path /models/DeepSeek-V2-Lite-Chat \ --gguf_path /models/DeepSeek-V2-Lite-Chat ``` <img width="951" height="486" alt="Image" src="https://github.com/user-attachments/assets/28bcd98d-bbcd-4d5d-b292-debbae0b5316" /> 可以正常进行对话，输入 ``` python -m ktransformers.server.main \ --model_path /models/DeepSeek-V2-Lite-Chat \ --gguf_path /models/DeepSeek-V2-Lite-Chat \ --port 11435 \ --host 0.0.0.0 ``` 可以正常开启服务 <img width="1101" height="178" alt="Image" src="https://github.com/user-attachments/assets/ca4ce413-0676-4cd5-b34a-ab45becf3dce" /> 使用chatbox或postman发送post请求 <img width="849" height="495" alt="Image" src="https://github.com/user-attachments/assets/eb2582fb-f229-4810-8bea-2d4c48f96e24" /> 出现错误 ``` INFO: 127.0.0.1:38502 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/workspace/ktransformers/ktransformers/server/backend/interfaces/transformers.py", line 227, in prepare_logits_wrapper self.model._get_logits_warper(generation_config, device=device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1931, in __getattr__ raise AttributeError( AttributeError: 'DeepseekV2ForCausalLM' object has no attribute '_get_logits_warper' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/conda/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/opt/conda/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__ await self.middleware_stack(scope, receive, send) File "/opt/conda/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__ raise exc File "/opt/conda/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__ await self.app(scope, receive, _send) File "/opt/conda/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__ await self.app(scope, receive, send) File "/opt/conda/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__ await self.middleware_stack(scope, receive, send) File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 734, in app await route.handle(scope, receive, send) File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/ktransformers/ktransformers/server/api/openai/endpoints/chat.py", line 436, in chat_completion async for res in interface.inference(input_message, id, create.temperature, create.top_p, create.max_tokens, create.max_completion_tokens): File "/workspace/ktransformers/ktransformers/server/backend/interfaces/ktransformers.py", line 241, in inference async for v in super().inference(local_messages, thread_id, temperature, top_p, max_tokens, max_completion_tokens): File "/workspace/ktransformers/ktransformers/server/backend/interfaces/transformers.py", line 412, in inference for t in self.prefill(input_ids, self.check_is_new(thread_id), temperature, top_p, max_tokens, max_completion_tokens): File "/opt/conda/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 36, in generator_context response = gen.send(None) ^^^^^^^^^^^^^^ File "/workspace/ktransformers/ktransformers/server/backend/interfaces/ktransformers.py", line 229, in prefill self.prepare_logits_wrapper(input_ids, device, temperature, top_p) File "/workspace/ktransformers/ktransformers/server/backend/interfaces/transformers.py", line 231, in prepare_logits_wrapper self.model._get_logits_warper(generation_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1931, in __getattr__ raise AttributeError( AttributeError: 'DeepseekV2ForCausalLM' object has no attribute '_get_logits_warper' ``` 请问这个如何解决呢？ ### Others _No response_

bsakaguchi

2026-10-29 14:48:16 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-29 14:48:16 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 8, 2025):

v0.3.1 已经放弃维护了，请用最新的 sglang+kt 的版本运行。另外，可以加我微信 ErvinXie 我拉你进我们的部署群，方便解决问题。

@ErvinXie commented on GitHub (Dec 8, 2025): v0.3.1 已经放弃维护了，请用最新的 sglang+kt 的版本运行。另外，可以加我微信 ErvinXie 我拉你进我们的部署群，方便解决问题。