Files
tabbyAPI/backends/exllamav2/model.py
kingbri aa34b2e5fd Model: Prefer auto over manual GPU split
For safety reasons, always use auto unless a manual split is provided
and auto is forced off.

If auto is forced off and a manual split isn't provided, a manual
split will be attempted.

Signed-off-by: kingbri <bdashore3@proton.me>
2024-02-17 00:21:48 -05:00

34 KiB