kingbri c0ad647fa7 Model: Auto-detect a one GPU setup and fix gpu_split_auto ...

It makes more sense to use gpu split parameters when the user has
>1 GPUs. Otherwise, set split and split_auto to False and save
the user some VRAM.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-02-06 23:08:57 -05:00

..

model.py

Model: Auto-detect a one GPU setup and fix gpu_split_auto

2024-02-06 23:08:57 -05:00

utils.py

Requirements: Update exllamav2, torch, and FA2

2024-02-02 23:53:42 -05:00