theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-22 23:38:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Sampling: Remove skip_special_tokens

Browse Source 
This parameter is way too confusing and does not make sense in
the modern LLM space.

Change approved by all maintainers.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri
2025-05-09 22:11:05 -04:00
parent 25c77ebf77
commit 42346c6b39
3 changed files with 0 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
backends/exllamav2/model.py
View File

@@ -1373,7 +1373,6 @@ class ExllamaV2Container(BaseModelContainer):
min_new_tokens=params.min_tokens,
gen_settings=gen_settings,
stop_conditions=stop_conditions,
decode_special_tokens=not params.skip_special_tokens,
filters=grammar_handler.filters,
filter_prefer_eos=bool(grammar_handler.filters),
return_probs=params.logprobs > 0,