ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-04-28 02:11:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
5cc8a89c08f49d08af58ba74ddf85d2bbb92e97e
ik_llama.cpp/ggml-cuda.cu
agray3 06151b32d7 Avoid unnecessarily disabling CUDA graphs (#7302) 
As discussed in PR #6766, CUDA graphs were being disabled in the presence of long prompts.
This fixes the issue by avoiding the consective update counter from incrementing unnecessarily
for tokens in which cuda graphs are disabled due to batch size > 1.
2024-05-15 15:44:49 +02:00

118 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink