mirror of
https://github.com/SillyTavern/SillyTavern-Tokenizers.git
synced 2026-10-15 18:12:02 +00:00
Tokenizer files to use with SillyTavern.
- Python 100%
| command-a-vision.json | ||
| command-a-vision.json.gz | ||
| command-a.json | ||
| command-a.json.gz | ||
| command-r.json | ||
| command-r.json.gz | ||
| deepseek.json | ||
| deepseek.json.gz | ||
| glm-4.6.json | ||
| glm-4.6.json.gz | ||
| LICENSE | ||
| minifier.py | ||
| nemo.json | ||
| nemo.json.gz | ||
| qwen2.json | ||
| qwen2.json.gz | ||
| README.md | ||
SillyTavern-Tokenizers
Tokenizer files to use with SillyTavern.
They will be automatically downloaded the first time you use the tokenizer from the app.
If you don't want to use internet downloads, download the JSON files and put them in the _cache subdirectory of your data root, /data/_cache by default. Create the _cache directory if you can't find it. After that, restart the SillyTavern server to re-initialize tokenizers.
Credits
All tokenizer data belongs to its owners. See the model licenses for more information.