ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-02-22 22:24:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

Forgot to add to fattn-common.h

Browse Source
This commit is contained in:
Kawrakow
2026-01-28 13:08:59 +00:00
parent 7aaed3f488
commit 6b37066b8f
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ggml/src/ggml-cuda/fattn-common.cuh
View File

@@ -17,6 +17,8 @@
#define HALF_MAX_HALF __float2half(65504.0f/2) // Use neg. of this instead of -INFINITY to initialize KQ max vals to avoid NaN upon subtraction.
#define SOFTMAX_FTZ_THRESHOLD -20.0f // Softmax exp. of values smaller than this are flushed to zero to avoid NaNs.
#define FATTN_KQ_MAX_OFFSET 1.386294361f
typedef void (* fattn_kernel_t)(
const char * __restrict__ Q,
const char * __restrict__ K,