[GH-ISSUE #456] 启动API服务以后，怎么在dify上接入模型？ #317

New issue

Closed

opened 2026-10-29 14:37:47 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:37:47 +00:00

Owner

Originally created by @yuweimian-shy on GitHub (Feb 18, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/456

我在后端启动了API服务：
USE_NUMA=1 python ./ktransformers/server/main.py --model_path /models/DeepSeek/DeepSeek-R1-Q4_K_M --gguf_path /models/DeepSeek/DeepSeek-R1-Q4_K_M --model_name DeepSeek-R1-Q4_K_M --cpu_infer 65 --max_new_tokens 2048 --port 9112 --cache_lens 12288

我想在dify平台上去接入这个模型，使用的是OpenAI-API-compatible

但一直有如下报错：
An error occurred during credentials validation: HTTPSConnectionPool(host='172.16.128.20', port=19112): Max retries exceeded with url: /v1/chat/completions (Caused by SSLError(SSLError(1, '[SSL] record layer failure (_ssl.c:1007)')))

用postman进行调用的时候是通的

Originally created by @yuweimian-shy on GitHub (Feb 18, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/456 我在后端启动了API服务： `USE_NUMA=1 python ./ktransformers/server/main.py --model_path /models/DeepSeek/DeepSeek-R1-Q4_K_M --gguf_path /models/DeepSeek/DeepSeek-R1-Q4_K_M --model_name DeepSeek-R1-Q4_K_M --cpu_infer 65 --max_new_tokens 2048 --port 9112 --cache_lens 12288` ![Image](https://github.com/user-attachments/assets/422d8a35-c0c0-4ea7-b127-92afc88f09d3) 我想在dify平台上去接入这个模型，使用的是OpenAI-API-compatible ![Image](https://github.com/user-attachments/assets/346a0681-ee81-4c13-aed5-484365402636) 但一直有如下报错： An error occurred during credentials validation: HTTPSConnectionPool(host='172.16.128.20', port=19112): Max retries exceeded with url: /v1/chat/completions (Caused by SSLError(SSLError(1, '[SSL] record layer failure (_ssl.c:1007)'))) 用postman进行调用的时候是通的

bsakaguchi

2026-10-29 14:37:47 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:37:48 +00:00

Author

Owner

@BlackHacked commented on GitHub (Feb 19, 2025):

你这明显没区分 http 和 https ，你映射出来的端口支持 https 么

@BlackHacked commented on GitHub (Feb 19, 2025): 你这明显没区分 http 和 https ，你映射出来的端口支持 https 么

bsakaguchi commented

2026-10-29 14:37:49 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！

bsakaguchi referenced this issue

2026-10-29 14:50:20 +00:00

[GH-ISSUE #2001] fixed: DeepSeek-V4-Flash outputs all-zero tokens on single RTX PRO 4000 Blackwell (SM_120, 24GB) #1230

bsakaguchi referenced this issue

2026-10-29 14:52:06 +00:00

[PR #317] [MERGED] [feature] update docker image and entrypoint #1402