Qianfeng 3d50f57f43 Update for fmha_fwd qs_ks_vs pipeline (#1810) ...

* Update for fmha_fwd qs_ks_vs pipeline

* Remove _builtin_amdgcn_sched_barrier(0)

* Move p_compute to p converting earlier for trying to increase vgprs re-using

* Enable GetQKBlockGemm to use WarpGemm-16x16x16 for QLoadOnce==false situation

* Re-add __builtin_amdgcn_sched_barrier(0)

---------

Co-authored-by: Po Yen Chen <PoYen.Chen@amd.com>

2025-01-13 12:43:05 +08:00

..

block_fmha_bwd_convert_dq.hpp

[CK_TILE] FA bwd kernels optimization (#1397)

2024-08-16 13:40:10 -07:00

block_fmha_bwd_dot_do_o.hpp

[CK_TILE] FA bwd kernels optimization (#1397)

2024-08-16 13:40:10 -07:00

block_fmha_bwd_dq_dk_dv_pipeline_kr_ktr_vr_iglp.hpp

[CK_TILE] Improve headdim96 performance for fmha-bwd (#1573)

2024-10-16 18:14:32 +08:00

block_fmha_bwd_dq_dk_dv_pipeline_kr_ktr_vr.hpp

[CK_TILE] Improve headdim96 performance for fmha-bwd (#1573)

2024-10-16 18:14:32 +08:00

block_fmha_bwd_pipeline_default_policy.hpp

[CK_TILE] Improve headdim96 performance for fmha-bwd (#1573)

2024-10-16 18:14:32 +08:00

block_fmha_bwd_pipeline_enum.hpp

[CK_TILE] FA bwd kernels optimization (#1397)

2024-08-16 13:40:10 -07:00

block_fmha_bwd_pipeline_problem.hpp

[CK_TILE] FA bwd kernels optimization (#1397)

2024-08-16 13:40:10 -07:00

block_fmha_fwd_appendkv_pipeline_default_policy.hpp

[CK_TILE] Add PagedAttention kernels (#1387)

2024-08-28 20:50:43 +08:00

block_fmha_fwd_appendkv_pipeline.hpp

[CK_TILE] Add PagedAttention kernels (#1387)

2024-08-28 20:50:43 +08:00

block_fmha_fwd_splitkv_combine_pipeline_default_policy.hpp

[CK_TILE] Add fmha fwd N-Warp S-Shuffle pipeline (fmha fwd splitkv pipeline variant) (#1705)

2024-12-20 14:41:01 +08:00

block_fmha_fwd_splitkv_combine_pipeline.hpp

roll back splitkv

2025-01-08 10:09:54 -08:00

block_fmha_fwd_splitkv_pipeline_nwarp_sshuffle_qr_ks_vs_default_policy.hpp

[CK_TILE] Add fmha fwd N-Warp S-Shuffle pipeline (fmha fwd splitkv pipeline variant) (#1705)

2024-12-20 14:41:01 +08:00

block_fmha_fwd_splitkv_pipeline_nwarp_sshuffle_qr_ks_vs.hpp

[CK_TILE] Add fmha fwd N-Warp S-Shuffle pipeline (fmha fwd splitkv pipeline variant) (#1705)

2024-12-20 14:41:01 +08:00

block_fmha_fwd_splitkv_pipeline_qr_ks_vs_default_policy.hpp

[CK_TILE] More fmha splitkv optimizations (#1588)

2024-10-26 18:35:45 +08:00

block_fmha_fwd_splitkv_pipeline_qr_ks_vs.hpp

roll back splitkv

2025-01-08 10:09:54 -08:00

block_fmha_pipeline_enum.hpp

[CK_TILE] support alibi (#1269)

2024-05-07 22:32:54 +08:00

block_fmha_pipeline_problem.hpp

[CK_TILE] fmha fwd splitkv optimization for decode (seqlen_q=1) (#1789)

2025-01-07 18:49:24 +08:00

block_fmha_pipeline_qr_ks_vs_async_default_policy.hpp

introducing ck_tile! (#1216)

2024-04-15 19:27:12 -05:00

block_fmha_pipeline_qr_ks_vs_async.hpp

[CK_TILE] fused-moe first version (#1634)

2024-11-26 11:14:56 +08:00

block_fmha_pipeline_qr_ks_vs_default_policy.hpp

introducing ck_tile! (#1216)

2024-04-15 19:27:12 -05:00

block_fmha_pipeline_qr_ks_vs_fp8.hpp

[CK_TILE] Add fmha fwd headdim96 support (#1608)

2024-10-30 14:03:16 +08:00

block_fmha_pipeline_qr_ks_vs.hpp

[CK_TILE] Add fmha fwd headdim96 support (#1608)

2024-10-30 14:03:16 +08:00

block_fmha_pipeline_qs_ks_vs_default_policy.hpp

Update for fmha_fwd qs_ks_vs pipeline (#1810)

2025-01-13 12:43:05 +08:00

block_fmha_pipeline_qs_ks_vs.hpp

Update for fmha_fwd qs_ks_vs pipeline (#1810)

2025-01-13 12:43:05 +08:00

block_fmha_pipeline_qx_ks_vs_custom_policy.hpp

Update for fmha_fwd qs_ks_vs pipeline (#1810)

2025-01-13 12:43:05 +08:00

tile_fmha_shape.hpp

[CK_TILE] Add fmha fwd N-Warp S-Shuffle pipeline (fmha fwd splitkv pipeline variant) (#1705)

2024-12-20 14:41:01 +08:00

tile_fmha_traits.hpp

[CK_TILE] fmha fwd splitkv optimization for decode (seqlen_q=1) (#1789)

2025-01-07 18:49:24 +08:00