Iwan Kawrakow 58d9e8f1d2 bitnet: put the scale in a separate tensor ...

and correspondingly add an extra ggml_mul_mat operation.
As per @ggerganov, this is how things should be done.
It seems to be working, but as far as I can tell this
results in a ~15% performance penalty for prompt processing.
Commiting so I can go and test on othe platforms.

2024-06-22 12:02:52 +03:00

12 KiB

Raw Blame History

View Raw