[PR #1158] [MERGED] Update Function call #1572

New issue

Closed

opened 2026-10-29 14:53:35 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:53:35 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/1158
Author: @Creeper-MZ
Created: 4/16/2025
Status: ✅ Merged
Merged: 4/19/2025
Merged by: @qiyuxinlin

Base: main ← Head: function_call

📝 Commits (6)

88f688e 更改token注入逻辑，减少token注入量，防止遗忘
6bc2e85 Update chat.py
cb266c9 Fix a bug
4fb19bf Update chat.py
62c4023 Fixed #1155
133ba74 优化提示词，解决部分Deepseek r1的兼容性

📊 Changes

2 files changed (+119 additions, -116 deletions)

View changed files

📝 ktransformers/server/api/openai/endpoints/chat.py (+117 -114)
📝 ktransformers/server/schemas/endpoints/chat.py (+2 -2)

📄 Description

Changed the injection logic for function information. Instead of injecting all the call information into the first prompt, it now injects the function call method into the first prompt, and then injects the function related information into the latest user message, which prevents the long context function call from being forgotten and optimizes the token usage. Changed the injection language to English.

Currently, the function call only supports Deepseek-v3, llama4 is not supported yet, after checking the chat template again, I found that llama4 doesn't need to inject messages, but needs to use a different method.

Only balance serve and deepseek v3 are tested.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/1158 **Author:** [@Creeper-MZ](https://github.com/Creeper-MZ) **Created:** 4/16/2025 **Status:** ✅ Merged **Merged:** 4/19/2025 **Merged by:** [@qiyuxinlin](https://github.com/qiyuxinlin) **Base:** `main` ← **Head:** `function_call` --- ### 📝 Commits (6) - [`88f688e`](https://github.com/kvcache-ai/ktransformers/commit/88f688e2c81f17a00128bd55ff4fc9536bbbcde9) 更改token注入逻辑，减少token注入量，防止遗忘 - [`6bc2e85`](https://github.com/kvcache-ai/ktransformers/commit/6bc2e85343b41df92cbc266ca6579090a0b17412) Update chat.py - [`cb266c9`](https://github.com/kvcache-ai/ktransformers/commit/cb266c98d41f047e0a0a1bd799588f56df8f6096) Fix a bug - [`4fb19bf`](https://github.com/kvcache-ai/ktransformers/commit/4fb19bfcae1f0e4960ae503589df313c675fa42b) Update chat.py - [`62c4023`](https://github.com/kvcache-ai/ktransformers/commit/62c40231606ad4df5452a207de647d75b00f7b23) Fixed #1155 - [`133ba74`](https://github.com/kvcache-ai/ktransformers/commit/133ba746e9fe490a1e99304195f6be7cca59e390) 优化提示词，解决部分Deepseek r1的兼容性 ### 📊 Changes **2 files changed** (+119 additions, -116 deletions) <details> <summary>View changed files</summary> 📝 `ktransformers/server/api/openai/endpoints/chat.py` (+117 -114) 📝 `ktransformers/server/schemas/endpoints/chat.py` (+2 -2) </details> ### 📄 Description Changed the injection logic for function information. Instead of injecting all the call information into the first prompt, it now injects the function call method into the first prompt, and then injects the function related information into the latest user message, which prevents the long context function call from being forgotten and optimizes the token usage. Changed the injection language to English. Currently, the function call only supports Deepseek-v3, llama4 is not supported yet, after checking the chat template again, I found that llama4 doesn't need to inject messages, but needs to use a different method. Only balance serve and deepseek v3 are tested. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:53:35 +00:00