theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-29 02:31:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Model: Fix tokenizer bugs

Browse Source 
Some tokenizer variables don't get cleaned up on init, so these can
persist. Clean these up manually before creating a new tokenizer for
now.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-03-20 11:23:17 -04:00
committed by Brian Dashore
parent 07d9b7cf7b
commit 7e669527ed
1 changed files with 6 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
backends/exllamav2/model.py
View File

@@ -390,7 +390,12 @@ class ExllamaV2Container:
# Notify that the model is being loaded # Notify that the model is being loaded
self.model_is_loading = True self.model_is_loading = True
# Load tokenizer # Reset tokenizer namespace vars and create a tokenizer
ExLlamaV2Tokenizer.unspecial_piece_to_id = {}
ExLlamaV2Tokenizer.unspecial_id_to_piece = {}
ExLlamaV2Tokenizer.extended_id_to_piece = {}
ExLlamaV2Tokenizer.extended_piece_to_id = {}
self.tokenizer = ExLlamaV2Tokenizer(self.config) self.tokenizer = ExLlamaV2Tokenizer(self.config)
# Calculate autosplit reserve for all GPUs # Calculate autosplit reserve for all GPUs