[PR #644] [MERGED] Allow temperature and top_p from requests #1442

New issue

Closed

opened 2026-10-29 14:52:27 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:52:27 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/644
Author: @wtdcode
Created: 2/24/2025
Status: ✅ Merged
Merged: 2/27/2025
Merged by: @qiyuxinlin

Base: main ← Head: temperature_top_p_from_request

📝 Commits (8)

8704c09 Allow temperature and top_p from requests
bf36547 Also allow repetition_penalty
05ad288 Also /chat/completions
76487c4 Revert repetition_penalty as it is not in API spec
91062a8 Default values
07eb712 Left out
26f7b4a Merge branch 'main' into temperature_top_p_from_request
b121ca4 Fix according to upstream changes

📊 Changes

6 files changed (+25 additions, -17 deletions)

View changed files

📝 ktransformers/server/api/openai/endpoints/chat.py (+2 -2)
📝 ktransformers/server/api/openai/legacy/completions.py (+2 -2)
📝 ktransformers/server/backend/interfaces/ktransformers.py (+5 -5)
📝 ktransformers/server/backend/interfaces/transformers.py (+11 -7)
📝 ktransformers/server/schemas/endpoints/chat.py (+3 -1)
📝 ktransformers/server/schemas/legacy/completions.py (+2 -0)

📄 Description

As title, this allows kTransformers to use temperature and top_p provided by requests instead of startup arguments. It is super inconvenient to restart to just change temperature.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/644 **Author:** [@wtdcode](https://github.com/wtdcode) **Created:** 2/24/2025 **Status:** ✅ Merged **Merged:** 2/27/2025 **Merged by:** [@qiyuxinlin](https://github.com/qiyuxinlin) **Base:** `main` ← **Head:** `temperature_top_p_from_request` --- ### 📝 Commits (8) - [`8704c09`](https://github.com/kvcache-ai/ktransformers/commit/8704c091923288d4ff37c37101ce617129df0d98) Allow temperature and top_p from requests - [`bf36547`](https://github.com/kvcache-ai/ktransformers/commit/bf36547f98bdf6b02264425c814f999219ab742f) Also allow repetition_penalty - [`05ad288`](https://github.com/kvcache-ai/ktransformers/commit/05ad2884535c1819603131badd212125d660a0ef) Also /chat/completions - [`76487c4`](https://github.com/kvcache-ai/ktransformers/commit/76487c4dcb09a2bfac93ac437ec0956325a50d07) Revert repetition_penalty as it is not in API spec - [`91062a8`](https://github.com/kvcache-ai/ktransformers/commit/91062a834f3b586beb58fe5ef20bb637ff8c3c27) Default values - [`07eb712`](https://github.com/kvcache-ai/ktransformers/commit/07eb712a73e65952114d3e404b4326d0d390cedc) Left out - [`26f7b4a`](https://github.com/kvcache-ai/ktransformers/commit/26f7b4af11d26c9b6f59d0542884e011afb609a6) Merge branch 'main' into temperature_top_p_from_request - [`b121ca4`](https://github.com/kvcache-ai/ktransformers/commit/b121ca4df8f921f254b14d28b3474e6d745ffed0) Fix according to upstream changes ### 📊 Changes **6 files changed** (+25 additions, -17 deletions) <details> <summary>View changed files</summary> 📝 `ktransformers/server/api/openai/endpoints/chat.py` (+2 -2) 📝 `ktransformers/server/api/openai/legacy/completions.py` (+2 -2) 📝 `ktransformers/server/backend/interfaces/ktransformers.py` (+5 -5) 📝 `ktransformers/server/backend/interfaces/transformers.py` (+11 -7) 📝 `ktransformers/server/schemas/endpoints/chat.py` (+3 -1) 📝 `ktransformers/server/schemas/legacy/completions.py` (+2 -0) </details> ### 📄 Description As title, this allows kTransformers to use `temperature` and `top_p` provided by requests instead of startup arguments. It is super inconvenient to restart to just change temperature. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:52:27 +00:00