turboderp-org/exllamav2
1
0
Fork 0
mirror of https://github.com/turboderp-org/exllamav2.git synced 2026-04-20 14:29:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
50 Commits 3 Branches 43 Tags
157fcfb5b95c98917afba9636a9c8a7bdb1e100b
Commit Graph

5 Commits

Author SHA1 Message Date
turboderp
918368b295 34B testing 2023-09-10 06:15:33 +02:00
turboderp
f79e16c5d0 Optimization, wider loads in EXL2 kernel (int4) 2023-09-07 10:56:43 +02:00
turboderp
c2f62e1f1f Optimization, wider loads in GPTQ kernel (int2) working 2023-09-07 04:07:13 +02:00
turboderp
3c80d41234 Add 4-bit GPTQ support 2023-09-05 14:03:51 +02:00
turboderp
a386102ac6 Improve prediction of VRAM usage when loading model 2023-09-01 10:47:29 +02:00