theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-14 15:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Model: Don't error out if a generation is empty

Browse Source 
When stream is false, the generation can be empty, which means
that there's no chunks present in the final generation array, causing
an error.

Instead, return a dummy value if generation is falsy (empty array
or None)

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2023-12-20 00:51:33 -05:00
parent ab10b263fd
commit 5728b9fffb
1 changed files with 6 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
model.py
View File

@@ -333,9 +333,12 @@ class ModelContainer:
return self.tokenizer.decode(ids, decode_special_tokens = unwrap(kwargs.get("decode_special_tokens"), True))[0]
def generate(self, prompt: str, **kwargs):
gen = list(self.generate_gen(prompt, **kwargs))
reponse = "".join(map(lambda o: o[0], gen))
return reponse, gen[-1][1], gen[-1][2]
generation = list(self.generate_gen(prompt, **kwargs))
if generation:
response = "".join(map(lambda chunk: chunk[0], generation))
return response, generation[-1][1], generation[-1][2]
else:
return "", 0, 0
def generate_gen(self, prompt: str, **kwargs):
"""