Model: Fix generation when stream = false

References #91. Check if the length of the generation array is > 0
after popping the finish reason.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-03-29 02:15:56 -04:00
parent d4280e1378
commit 190a0b26c3

View File

@@ -602,6 +602,7 @@ class ExllamaV2Container:
else:
joined_generation["finish_reason"] = "stop"
if len(generations) > 0:
for generation in generations:
joined_generation["text"] += unwrap(generation.get("text"), "")
joined_generation["offset"].append(unwrap(generation.get("offset"), -1))