ROCm/composable_kernel
1
0
Fork 0
mirror of https://github.com/ROCm/composable_kernel.git synced 2026-07-12 02:05:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Adjust the threhold value in shall_use_splitkv()

Browse Source
This commit is contained in:
Qianfeng Zhang
2026-06-30 09:09:04 +00:00
parent 26400def37
commit 80d7b8dce7
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
example/ck_tile/18_hstu_attention/hstu_attention_splitkv_helper.hpp
View File

@@ -50,7 +50,7 @@ get_estimated_cu_coverage_ratio(int num_batches, int num_heads, int max_seqlen_q
static bool shall_use_splitkv(int num_batches, int num_heads, int max_seqlen_q, int max_seqlen_kv)
{
// Please tune the threshold here
const float threshold = (max_seqlen_kv >= 2048) ? 1.5f : 0.8f;
const float threshold = (max_seqlen_kv >= 8192) ? 3.0f : 0.8f;
if(get_estimated_cu_coverage_ratio(num_batches, num_heads, max_seqlen_q, max_seqlen_kv) <
threshold)