mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #578] 通过 Open-WebUI 使用 KTransformer #425
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#425
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @TTThanos on GitHub (Feb 21, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/578
请问这个无法选择模型是什么情况？
外部链接已经设置成 http://localhost:8000/v1
后端kTransformer服务启动命令：
USE_NUMA=1 python3 ./ktransformers/server/main.py --model_path /root/DeepSeek-R1-GGUF --gguf_path /root/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M/ --model_name deepseek_r1 --cpu_infer 65 --max_new_tokens 1000 --port 8000 --cache_lens 12288 --force_think true
WebUI docker创建命令：
docker run -it --net=host --ipc=host
-e ENABLE_OLLAMA_API=False
-e 0PENAI_API_BASE_URL=http://127.0.0.1:8000/v1
--name open-webui
ghcr.io/open-webui/open-webui:main
@manhere commented on GitHub (Feb 21, 2025):
应该是没实现模型列表接口
@yeungtuzi commented on GitHub (Feb 21, 2025):
docker 里面访问不能使用localhost，注意kt要监听0.0.0.0，然后open-webui里面访问 http://host.docker.internal:8000/v1
@ginghalo commented on GitHub (Feb 21, 2025):
换成openai api ，然后再在
ktransformers_root/ktransformers/server/api/openai/endpoints/chat.py里改成comp = ChatCompletionObject(id=id,object='chat.completion',created=int(time()))。因为open-webui默认用的不是stream，而非stream模式在ollama api里目前不支持，且目前的openai api的非steam模式也有bug，要改一下。
@fangfenghuang commented on GitHub (Feb 24, 2025):
可以提供更多的对接细节吗？我这边改了后还是不行
@ginghalo commented on GitHub (Feb 25, 2025):
我换成dify了，用open-webui有时可以识别有时不行。
@luna0804 commented on GitHub (Feb 25, 2025):
你好，请问怎么安装dify呢，也是要装docker吗?
@ginghalo commented on GitHub (Feb 25, 2025):
可以看看(repo)[https://github.com/langgenius/dify/blob/main/README_CN.md]，模型就选择兼容openai模型，链接就是
http://localhost:port/v1，选择非stream。@TTThanos commented on GitHub (Feb 25, 2025):
https://github.com/ubergarm/r1-ktransformers-guide/blob/main/README.zh.md
参考此文档即可使用WebUI访问部署的DeepSeek，需要把kTransformers 拉到最新版本
@luna0804 commented on GitHub (Feb 26, 2025):
谢谢，我现在是在Linux服务器上装的ktransformer，在命令行可以进行对话，现在想开启web服务在网页打开。我安装好了openwebui，也搞好了网络代理可以在本机电脑打开openwebui界面，但是识别不到服务器里的deepseek模型......服务器里没安装docker和ollama，请问你有什么建议吗？
@ErvinXie commented on GitHub (Dec 11, 2025):
This issue has been automatically closed due to inactivity for 60 days.
If you believe this issue is still relevant, please feel free to reopen it with additional information or context.
Thank you for your contribution to KTransformers!
此 issue 因超过 60 天无活动而被自动关闭。
如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。
感谢您对 KTransformers 的贡献！