[PR #1137] [CLOSED] Refactor the chat interface to support tool calling and parameter pro… #1569

New issue

Closed

opened 2026-10-29 14:53:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:53:34 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/1137
Author: @344303947
Created: 4/14/2025
Status: ❌ Closed

Base: main ← Head: features/add-function-calling

📝 Commits (1)

8699109 Refactor the chat interface to support tool calling and parameter processing

📊 Changes

6 files changed (+574 additions, -99 deletions)

View changed files

📝 ktransformers/server/api/openai/endpoints/chat.py (+392 -65)
📝 ktransformers/server/backend/interfaces/balance_serve.py (+19 -9)
📝 ktransformers/server/backend/interfaces/ktransformers.py (+3 -2)
📝 ktransformers/server/backend/interfaces/transformers.py (+115 -10)
📝 ktransformers/server/config/config.py (+1 -1)
📝 ktransformers/server/schemas/endpoints/chat.py (+44 -12)

📄 Description

…cessing

Defined new data structures in chat.py to replace OpenAI's original implementation, adding support for tool calling.

Implemented logic for extracting and processing tool calls, enabling dynamic function invocation during conversations.

Added methods in balance_serve.py to retrieve sampling parameters, handling default values and edge cases.

Updated ktransformers.py and transformers.py to support the passing of tool parameters.

Modified the default value of top_p in config.py to 1.0 to increase generation diversity.

Extended the message model in chat.py to support the transmission of tool call information.

These changes enhance the system's flexibility and functionality, enabling more complex interaction patterns.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/1137 **Author:** [@344303947](https://github.com/344303947) **Created:** 4/14/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `features/add-function-calling` --- ### 📝 Commits (1) - [`8699109`](https://github.com/kvcache-ai/ktransformers/commit/869910912929bc5c4a8493e726e0a9653d6073cd) Refactor the chat interface to support tool calling and parameter processing ### 📊 Changes **6 files changed** (+574 additions, -99 deletions) <details> <summary>View changed files</summary> 📝 `ktransformers/server/api/openai/endpoints/chat.py` (+392 -65) 📝 `ktransformers/server/backend/interfaces/balance_serve.py` (+19 -9) 📝 `ktransformers/server/backend/interfaces/ktransformers.py` (+3 -2) 📝 `ktransformers/server/backend/interfaces/transformers.py` (+115 -10) 📝 `ktransformers/server/config/config.py` (+1 -1) 📝 `ktransformers/server/schemas/endpoints/chat.py` (+44 -12) </details> ### 📄 Description …cessing Defined new data structures in chat.py to replace OpenAI's original implementation, adding support for tool calling. Implemented logic for extracting and processing tool calls, enabling dynamic function invocation during conversations. Added methods in balance_serve.py to retrieve sampling parameters, handling default values and edge cases. Updated ktransformers.py and transformers.py to support the passing of tool parameters. Modified the default value of top_p in config.py to 1.0 to increase generation diversity. Extended the message model in chat.py to support the transmission of tool call information. These changes enhance the system's flexibility and functionality, enabling more complex interaction patterns. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:53:34 +00:00