theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-20 14:28:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Model: Remove generate_window

Browse Source 
Not required since we error with exceeding the max_seq_len

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri
2025-04-16 12:58:23 -04:00
parent 2f5235e1a3
commit 1afc9b983e
3 changed files with 0 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
backends/exllamav2/model.py
View File

@@ -1242,9 +1242,6 @@ class ExllamaV2Container:
grammar_handler = ExLlamaV2Grammar()
banned_strings = []
# TODO: Not used for some reason?
generate_window = max(params.generate_window, self.config.max_seq_len // 8)
self.assign_gen_params(
params,
gen_settings,