Commit Graph

3 Commits

Author SHA1 Message Date
turboderp
4afe616aee Fix unhandled OoM condition when loading GPTQ model with auto split
Free minimum reserved VRAM on previous device when moving to next device
2023-10-28 20:08:39 +02:00
turboderp
093b89d38c Add generator versions of model.load() and model.load_autosplit() 2023-10-23 01:17:10 +02:00
turboderp
eb2cae6c52 Add auto GPU split feature 2023-10-22 18:48:35 +02:00