turboderp-org/exllamav2
1
0
Fork 0
mirror of https://github.com/turboderp-org/exllamav2.git synced 2026-05-04 05:01:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
601 Commits 3 Branches 43 Tags
f7c89f4c5139207400a4c72982f4e3d3b15172d9
Commit Graph

5 Commits

Author SHA1 Message Date
turboderp
d8b4efa8d4 Instrumentation etc. 2023-12-10 17:36:40 +01:00
turboderp
5834f3a968 Make sure all inference is done in torch.inference_mode() 2023-10-22 20:23:42 +02:00
turboderp
f79e16c5d0 Optimization, wider loads in EXL2 kernel (int4) 2023-09-07 10:56:43 +02:00
turboderp
c2f62e1f1f Optimization, wider loads in GPTQ kernel (int2) working 2023-09-07 04:07:13 +02:00
turboderp
3c80d41234 Add 4-bit GPTQ support 2023-09-05 14:03:51 +02:00