Kawrakow ff718c2dc1 Bitnet(2.25 bpw): CUDA ...

We get PP-512 = 9600 t/s, TG-128 = 234 t/s
(but we need to use 8 CPU threads, else results are lower,
so clearly there is something being computed on the CPU).
PP-512 is very close to PP-512(fp16) = 9800 t/s

2024-06-22 12:02:52 +03:00

119 KiB

Raw Blame History

View Raw