Anton Gorenko e9596228ff Fix synchronization issue in fwd qr pipeline with dropout (#3135) ...

BlockFmhaPipelineQRKSVS reuses LDS for K and dropout so there must be
block_sync_lds between loading k_lds_window by gemm_0 and storing
dropout randval.

2025-10-31 09:44:52 -07:00

..

block_fmha_batch_prefill_pipeline_qr_ks_vs_async_default_policy.hpp

[CK_TILE] Add logits soft-capping & customization support to the FMHA forward kernel/pipelines (#2163)

2025-05-13 12:19:25 +08:00

block_fmha_batch_prefill_pipeline_qr_ks_vs_async.hpp

Revert "Eliminate warning caused by failed to meet occupancy requirement (#2389)" (#2514)

2025-07-17 10:09:01 +08:00

block_fmha_bwd_convert_dq.hpp

[CK_TILE] FA bwd kernels optimization (#1397)

2024-08-16 13:40:10 -07:00

block_fmha_bwd_dot_do_o.hpp

[CK_TILE] FA bwd kernels optimization (#1397)

2024-08-16 13:40:10 -07:00

block_fmha_bwd_dq_dk_dv_pipeline_kr_ktr_vr_iglp.hpp

[CK_TILE] FMHA BWD Pad HDim to a Multiple of 8 (#2918)

2025-09-26 16:42:59 +08:00

block_fmha_bwd_dq_dk_dv_pipeline_kr_ktr_vr.hpp

[CK_TILE] FMHA BWD Pad HDim to a Multiple of 8 (#2918)

2025-09-26 16:42:59 +08:00

block_fmha_bwd_dq_dk_dv_pipeline_selector.hpp

[CK_TILE] FMHA BWD Pad HDim to a Multiple of 8 (#2918)

2025-09-26 16:42:59 +08:00

block_fmha_bwd_dq_dk_dv_pipeline_trload_kr_ktr_vr.hpp

[CK_TILE] FMHA BWD Pad HDim to a Multiple of 8 (#2918)

2025-09-26 16:42:59 +08:00

block_fmha_bwd_dq_dk_dv_pipeline_trload_qr_qtr_dor.hpp

[CK_TILE] FMHA BWD Pad HDim to a Multiple of 8 (#2918)

2025-09-26 16:42:59 +08:00

block_fmha_bwd_pipeline_default_policy.hpp

[CK_TILE] Support WMMA (gfx12) in FMHA (#2528)

2025-10-29 13:31:08 -07:00

block_fmha_bwd_pipeline_problem.hpp

[CK_TILE] FMHA BWD Pad HDim to a Multiple of 8 (#2918)

2025-09-26 16:42:59 +08:00

block_fmha_bwd_pipeline_trload_default_policy.hpp

Reapply "[CK_TILE] FMHA BWD Enable Tile 16x192 (#2741)" (#2757) (#2761)

2025-09-08 09:21:14 -07:00

block_fmha_fwd_appendkv_pipeline_default_policy.hpp

[CK_TILE] Add PagedAttention kernels (#1387)

2024-08-28 20:50:43 +08:00

block_fmha_fwd_appendkv_pipeline.hpp

[CK_TILE] Add PagedAttention kernels (#1387)

2024-08-28 20:50:43 +08:00

block_fmha_fwd_pagedkv_pipeline_qr_ks_vs_default_policy.hpp

[CK_TILE] Support WMMA (gfx12) in FMHA (#2528)

2025-10-29 13:31:08 -07:00

block_fmha_fwd_pagedkv_pipeline_qr_ks_vs.hpp

Fix for Add the API to load SGPR (#2913)

2025-09-25 10:32:42 -07:00

block_fmha_fwd_splitkv_combine_pipeline_default_policy.hpp

[CK_TILE] Support WMMA (gfx12) in FMHA (#2528)

2025-10-29 13:31:08 -07:00

block_fmha_fwd_splitkv_combine_pipeline.hpp

[CK_TILE] Support WMMA (gfx12) in FMHA (#2528)

2025-10-29 13:31:08 -07:00

block_fmha_fwd_splitkv_pipeline_nwarp_sshuffle_qr_ks_vs_default_policy.hpp

Ck tile/complete k prefetch (#1941)

2025-03-07 14:19:51 +08:00

block_fmha_fwd_splitkv_pipeline_nwarp_sshuffle_qr_ks_vs.hpp

[CK_TILE] Add logits soft-capping & customization support to the FMHA forward kernel/pipelines (#2163)

2025-05-13 12:19:25 +08:00

block_fmha_fwd_splitkv_pipeline_qr_ks_vs_default_policy.hpp

Ck tile/complete k prefetch (#1941)

2025-03-07 14:19:51 +08:00

block_fmha_fwd_splitkv_pipeline_qr_ks_vs.hpp

Fix for Add the API to load SGPR (#2913)

2025-09-25 10:32:42 -07:00

block_fmha_fwd_v3_pipeline_default_policy.hpp

[CK_TILE] Add FAv3 fwd pipeline (#2731)

2025-09-01 09:16:45 +08:00

block_fmha_fwd_v3_pipeline.hpp

[CK_TILE] FMHA FAv3 scheduling fine-tuning for performance (#2833)

2025-09-16 11:32:38 +08:00

block_fmha_pipeline_enum.hpp

Re-enable optimization for gfx950 fmha fwd (#2671)

2025-08-13 14:57:43 +08:00

block_fmha_pipeline_problem.hpp

[CK_TILE] Support WMMA (gfx12) in FMHA (#2528)

2025-10-29 13:31:08 -07:00

block_fmha_pipeline_qr_ks_vs_async_default_policy.hpp

Ck tile/complete k prefetch (#1941)

2025-03-07 14:19:51 +08:00

block_fmha_pipeline_qr_ks_vs_async_trload_policy.hpp

[CK_TILE] Support WMMA (gfx12) in FMHA (#2528)

2025-10-29 13:31:08 -07:00

block_fmha_pipeline_qr_ks_vs_async_trload.hpp

[CKTILE] FMHA fwd trload lse fix (#3046)

2025-10-23 09:33:33 +08:00

block_fmha_pipeline_qr_ks_vs_async.hpp

Fix fmha fwd precision issue on MI3XX series (#2285)

2025-06-10 15:03:23 +08:00

block_fmha_pipeline_qr_ks_vs_default_policy.hpp

Ck tile/complete k prefetch (#1941)

2025-03-07 14:19:51 +08:00

block_fmha_pipeline_qr_ks_vs_fp8.hpp

[CK_TILE] Add fmha fwd headdim96 support (#1608)

2024-10-30 14:03:16 +08:00

block_fmha_pipeline_qr_ks_vs_whole_k_prefetch_default_policy.hpp

[CK_TILE] Support WMMA (gfx12) in FMHA (#2528)

2025-10-29 13:31:08 -07:00

block_fmha_pipeline_qr_ks_vs_whole_k_prefetch.hpp

Update for xformers (#2372)

2025-06-22 00:28:30 -07:00

block_fmha_pipeline_qr_ks_vs.hpp

Fix synchronization issue in fwd qr pipeline with dropout (#3135)

2025-10-31 09:44:52 -07:00

block_fmha_pipeline_qs_ks_vs_default_policy.hpp

Ck tile/complete k prefetch (#1941)

2025-03-07 14:19:51 +08:00

block_fmha_pipeline_qs_ks_vs.hpp

[CK_TILE] Add logits soft-capping & customization support to the FMHA forward kernel/pipelines (#2163)

2025-05-13 12:19:25 +08:00

block_fmha_pipeline_qx_ks_vs_custom_policy.hpp

[CK_TILE] Support WMMA (gfx12) in FMHA (#2528)

2025-10-29 13:31:08 -07:00

tile_fmha_shape.hpp

[CK_TILE] Support f32 in FMHA (fwd and bwd) (#2836)

2025-09-27 18:03:48 +05:00

tile_fmha_traits.hpp

[CK_TILE] FMHA BWD Pad HDim to a Multiple of 8 (#2918)

2025-09-26 16:42:59 +08:00