mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #267] start with website fail when using DeepSeek-R1-Q4_K_M #167
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#167
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @neichuanzhang on GitHub (Feb 14, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/267
When i trying to start the deepseek with website, the error goes like:
(ktransformers2) root@xx:~# ktransformers --model_path deepseek-ai/DeepSeek-R1 --gguf_path /root/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M/ --port 10002 --web True 2025-02-14 05:53:17,236 INFO /root/miniconda3/envs/ktransformers2/lib/python3.11/site-packages/ktransformers/server/main.py[25]: Creating SQL tables
2025-02-14 05:53:17,238 INFO /root/miniconda3/envs/ktransformers2/lib/python3.11/site-packages/ktransformers/server/api/openai/assistants/assistants.py[75]: Creating default assistant
Traceback (most recent call last):
File "/root/miniconda3/envs/ktransformers2/bin/ktransformers", line 8, in
sys.exit(main())
^^^^^^
File "/root/miniconda3/envs/ktransformers2/lib/python3.11/site-packages/ktransformers/server/main.py", line 134, in main
create_interface(config=cfg, default_args=default_args)
File "/root/miniconda3/envs/ktransformers2/lib/python3.11/site-packages/ktransformers/server/utils/create_interface.py", line 27, in create_interface
GlobalInterface.interface = BackendInterface(default_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/ktransformers2/lib/python3.11/site-packages/ktransformers/server/backend/interfaces/ktransformers.py", line 27, in init
self.model=custom_modelsconfig.architectures[0]
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'DeepseekV3ForCausalLM'
while if I start the deepseek without website, via the command:
python -m ktransformers.local_chat --model_path deepseek-ai/DeepSeek-R1 --gguf_path /root/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M/ --cpu_infer 65 --max_new_tokens 1000
it start successfull, so what can i do to start the deepseek with website.
besides, I try to install ktransformers via pip install ktransformers --no-build-isolation and download source code and compile, both do not work.
@Wilbur0626 commented on GitHub (Feb 14, 2025):
the same error,seeking solutions
@Czhang-zc commented on GitHub (Feb 14, 2025):
+1
@Wilbur0626 commented on GitHub (Feb 14, 2025):
solved, please upgrade the version of ktransformers
https://github.com/kvcache-ai/ktransformers/releases
@ErvinXie commented on GitHub (Dec 11, 2025):
This issue has been automatically closed due to inactivity for 60 days.
If you believe this issue is still relevant, please feel free to reopen it with additional information or context.
Thank you for your contribution to KTransformers!
此 issue 因超过 60 天无活动而被自动关闭。
如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。
感谢您对 KTransformers 的贡献！