theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-19 22:08:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
09a4c79847867cd1868a23171eaeac7aefadb76d
tabbyAPI/backends
History
kingbri 09a4c79847 Model: Auto-scale max_tokens by default 
If max_tokens is None, it automatically scales to fill up the context.
This does not mean the generation will fill up that context since
EOS stops also exist.

Originally suggested by #86

Signed-off-by: kingbri <bdashore3@proton.me>
2024-03-18 22:54:59 -04:00
..
exllamav2
Model: Auto-scale max_tokens by default
2024-03-18 22:54:59 -04:00