theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-15 00:07:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix: Properly handle banned_strings and decode_special tokens (#104)

Browse Source 
* Fix: Actually pass banned_strings to the generation call.

* decode_special_tokens was missing as well.

* syntax
This commit is contained in:
Ycros
2024-05-13 06:47:45 +10:00
committed by GitHub
parent 611f00818b
commit 57525219d0
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
backends/exllamav2/model.py
View File

@@ -919,6 +919,8 @@ class ExllamaV2Container:
"return_top_tokens": request_logprobs,
"return_logits": request_logprobs > 0,
"abort_event": abort_event,
"banned_strings": banned_strings,
"decode_special_tokens": decode_special_tokens,
}
if self.use_cfg: