root e8587b86c2 Fix CK-UA pipeline: s_waitcnt_vmcnt<0> in fmha_post_process ...

The final V tile's async load was not properly waited on before reading
from LDS: s_waitcnt_vmcnt<K_inst> allowed V_inst outstanding loads
(a no-op when K_inst == V_inst). The last loop iteration never prefetches
K, so only V is outstanding. Use s_waitcnt_vmcnt<0> unconditionally.

This partially fixes the BS32 race condition for production workloads
(maxk >= 256). A deeper pipeline race remains for very short KV
sequences (maxk < ~165, 2-5 pages) with block_size=32 at high batch.

Made-with: Cursor

2026-04-01 23:04:07 +00:00

..

add_rmsnorm2d_rdquant

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

batched_contraction

[CK Tile] batched contraction kernel generalizing (#3126)

2025-12-02 13:30:27 +01:00

batched_transpose

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

common

[rocm-libraries] ROCm/rocm-libraries#5393 (commit d51b649)

2026-03-27 09:18:14 +00:00

elementwise

[rocm-libraries] ROCm/rocm-libraries#5237 (commit ef10dc6)

2026-03-13 01:21:08 +00:00

epilogue

[rocm-libraries] ROCm/rocm-libraries#5729 (commit 516c974)

2026-03-31 03:40:25 +00:00

flatmm

[rocm-libraries] ROCm/rocm-libraries#5045 (commit 64a5502)

2026-03-03 21:55:14 +00:00

fmha

Fix fmha_fwd early-exit bug: seqlen_q <= min_seqlen_q should be <

2026-04-01 16:24:31 +00:00

fused_moe

[rocm-libraries] ROCm/rocm-libraries#4819 (commit b995a0b)

2026-02-25 16:13:13 +00:00

gemm

[rocm-libraries] ROCm/rocm-libraries#5776 (commit ee1bbcb)

2026-04-01 16:22:08 +00:00

gemm_mx

[rocm-libraries] ROCm/rocm-libraries#5849 (commit d9b89b2)

2026-03-27 20:37:23 +00:00

gemm_quant

[rocm-libraries] ROCm/rocm-libraries#5323 (commit 5454e9e)

2026-03-17 18:58:56 +00:00

grouped_convolution

[rocm-libraries] ROCm/rocm-libraries#5842 (commit 04c5690)

2026-03-31 08:03:41 +00:00

image_to_column

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

layernorm2d

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

norm_reduce

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

permute

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

pooling

Shuffle fix for gfx950 (#3491)

2026-01-13 09:21:29 -08:00

reduce

[CK Tile] multi reduce improvements (#3607)

2026-01-27 12:56:09 -08:00

rmsnorm2d

Fix redundant cast in model sensitive rmsnorm (#3681)

2026-01-30 10:52:19 +08:00

smoothquant

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

softmax/block

[rocm-libraries] ROCm/rocm-libraries#4274 (commit 7c380df)

2026-02-11 05:52:42 +00:00

sparse_attn

[CK_TILE][FMHA] Add sparse attention VSA (#3341)

2026-01-31 00:59:47 +08:00

topk/block

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

topk_softmax

Shuffle fix for gfx950 (#3491)

2026-01-13 09:21:29 -08:00

unified_attention

Fix CK-UA pipeline: s_waitcnt_vmcnt<0> in fmha_post_process

2026-04-01 23:04:07 +00:00

add_rmsnorm2d_rdquant.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

batched_contraction.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

batched_transpose.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

common.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

elementwise.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

epilogue.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

flatmm.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

fmha.hpp

[rocm-libraries] ROCm/rocm-libraries#4368 (commit 17f7dfc)

2026-03-11 10:00:52 +00:00

fused_moe.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

gemm_mx.hpp

[rocm-libraries] ROCm/rocm-libraries#5241 (commit 43daeac)

2026-03-12 08:27:49 +00:00

gemm_quant.hpp

[rocm-libraries] ROCm/rocm-libraries#4964 (commit 3271d9a)

2026-03-16 08:31:56 +00:00

gemm.hpp

[rocm-libraries] ROCm/rocm-libraries#4964 (commit 3271d9a)

2026-03-16 08:31:56 +00:00

grouped_convolution.hpp

[rocm-libraries] ROCm/rocm-libraries#5241 (commit 43daeac)

2026-03-12 08:27:49 +00:00

image_to_column.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

layernorm2d.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

moe_flatmm.hpp

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

norm_reduce.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

permute.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

pooling.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

reduce.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

rmsnorm2d.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

smoothquant.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

softmax.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

sparse_attn.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

topk_softmax.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

topk.hpp

[rocm-libraries] ROCm/rocm-libraries#4294 (commit 6601702)

2026-03-02 12:21:44 +00:00

unified_attention.hpp

Add unified attention (42_unified_attention)

2026-04-01 16:39:15 +00:00