turboderp-org/exllamav2
1
0
Fork 0
mirror of https://github.com/turboderp-org/exllamav2.git synced 2026-05-04 05:01:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
601 Commits 3 Branches 43 Tags
f7c89f4c5139207400a4c72982f4e3d3b15172d9
Commit Graph

3 Commits

Author SHA1 Message Date
turboderp
4afe616aee Fix unhandled OoM condition when loading GPTQ model with auto split 
Free minimum reserved VRAM on previous device when moving to next device
 2023-10-28 20:08:39 +02:00
turboderp
093b89d38c Add generator versions of model.load() and model.load_autosplit() 2023-10-23 01:17:10 +02:00
turboderp
eb2cae6c52 Add auto GPU split feature 2023-10-22 18:48:35 +02:00