ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-01-26 17:20:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
ik_llama.cpp/github-data/issues/217 - Bug_ CPU FA with fp16 K-cache is broken.md
Thomas eaa2510a28 Add GitHub data: filename sanitization (#640)
2025-07-23 13:31:53 +02:00

591 B
Raw Permalink Blame History

🐛 #217 - Bug: CPU FA with fp16 K-cache is broken

Author ikawrakow
State ❌ Closed
Created 2025-02-21
Updated 2025-02-22

Description

What happened?

Running HellaSwag with flash attention enabled and using fp16 for K-cache produces much lower scores than no FA or FA using Q8_0 or bf16 for K-cache.

Name and Version

Latest

What operating system are you seeing the problem on?

No response

Relevant log output