theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-20 14:28:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
5bb4995a7c6d0d8fa486b4d4c3a3e062155afde5
tabbyAPI/backends/exllamav2/model.py
kingbri b11aac51e2 Model: Add torch.inference_mode() to generator function 
Provides a speedup to model forward.

Signed-off-by: kingbri <bdashore3@proton.me>
2024-03-30 10:45:28 -04:00

37 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink