ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-01-26 09:09:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
ik_llama.cpp/github-data/pull_requests/349 - Fix division by zero bug.md
Thomas eaa2510a28 Add GitHub data: filename sanitization (#640)
2025-07-23 13:31:53 +02:00

897 B
Raw Permalink Blame History

🐛 #349 - Fix division by zero bug

Author ikawrakow
State ❌ Closed
Created 2025-04-26
Updated 2025-04-26

Description

The bug was in the calculation of number of work items to use when computing FA on the CPU. In my case (maximum of 32 threads) it triggered with the GLM-4 model that has an unusually small number of KV heads (just 2). But I guess it can also trigger with a larger number of threads for more common numbers of KV heads.

Fixed by just using max(1, nk). This will result in a far from optimal number of compute chunks, but at least it works.

I'm working on a better strategy for dividing the work between the threads on this branch, but not quite ready for a PR yet.