turboderp 3c80d41234 Add 4-bit GPTQ support

2023-09-05 14:03:51 +02:00

..

chat.py

Add 4-bit GPTQ support

2023-09-05 14:03:51 +02:00

inference.py

Reworking attention, allow for batched inference with independent cache per sequence

2023-09-03 15:56:38 +02:00

streaming.py

Tidying up

2023-09-02 16:40:57 +02:00