[PR #77] [MERGED] Fix: Wrong type of token list returned by prefill_and_generate #1344

New issue

Closed

opened 2026-10-29 14:51:36 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:51:36 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/77
Author: @TKONIY
Created: 9/5/2024
Status: ✅ Merged
Merged: 10/9/2024
Merged by: @UnicornChan

Base: main ← Head: fix-prefill-and-generate

📝 Commits (1)

ee72cee Fix: the tokens return by prefill_and_generate

📊 Changes

1 file changed (+2 additions, -2 deletions)

View changed files

📝 ktransformers/util/utils.py (+2 -2)

📄 Description

Bug Report

The prefill_and_generate returns List[Tensor], which cannot be fed into tokenizer.decode. e.g., [Tensor[1], Tensor[2]]

Fix

Make it returns List[int], e.g., [1, 2]

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/77 **Author:** [@TKONIY](https://github.com/TKONIY) **Created:** 9/5/2024 **Status:** ✅ Merged **Merged:** 10/9/2024 **Merged by:** [@UnicornChan](https://github.com/UnicornChan) **Base:** `main` ← **Head:** `fix-prefill-and-generate` --- ### 📝 Commits (1) - [`ee72cee`](https://github.com/kvcache-ai/ktransformers/commit/ee72cee050d6ccfd9ab8ed58c24f00e3b2eff7df) Fix: the tokens return by prefill_and_generate ### 📊 Changes **1 file changed** (+2 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `ktransformers/util/utils.py` (+2 -2) </details> ### 📄 Description ## Bug Report The `prefill_and_generate` returns `List[Tensor]`, which cannot be fed into `tokenizer.decode`. e.g., `[Tensor[1], Tensor[2]]` ## Fix Make it returns `List[int]`, e.g., `[1, 2]` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:51:36 +00:00