theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-14 15:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
0985c7f7b7b7847078f757867cae8dfeebba2fe9
tabbyAPI/config_sample.yml
kingbri 5cb8f3ed2c Config: Fix comments for max_seq_len and cache_size 
The default is the minimum between max_position_embeddings and cache_size.
On AMD and older than Ampere NVIDIA GPUs, cache_size is ignored due
to not being supported by batching on exl2.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
2025-10-14 23:04:36 -04:00

8.5 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink