turboderp-org/exllamav2
1
0
Fork 0
mirror of https://github.com/turboderp-org/exllamav2.git synced 2026-05-04 21:21:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,030 Commits 3 Branches 43 Tags
c2aac982e4e2b01ae022e9d6a8a4cd908afb4e4b
Commit Graph

5 Commits

Author SHA1 Message Date
turboderp
d8b4efa8d4 Instrumentation etc. 2023-12-10 17:36:40 +01:00
turboderp
5834f3a968 Make sure all inference is done in torch.inference_mode() 2023-10-22 20:23:42 +02:00
turboderp
f79e16c5d0 Optimization, wider loads in EXL2 kernel (int4) 2023-09-07 10:56:43 +02:00
turboderp
c2f62e1f1f Optimization, wider loads in GPTQ kernel (int2) working 2023-09-07 04:07:13 +02:00
turboderp
3c80d41234 Add 4-bit GPTQ support 2023-09-05 14:03:51 +02:00