Ye Wang d7bb3b10cc [fmha-bwd] Flat cu_id remap for arbitrary CTA_NUM + grid Y/Z override env ...

The persistent bwd group kernel remapped block->CU as bx/8 + (bx%8)*32,
which is only bijective for N=256 -> wrong gradients when ROCM_FLASH_ATTN_CU_NUM
is set to any other value. Replace with flat blockIdx (bx + by*gridDim.x +
bz*gridDim.x*gridDim.y), a bijection for any 1D/3D grid. Also add
get_persistent_grid_override() honoring ROCM_FLASH_ATTN_GRID_Y/Z to reshape
the persistent grid for 1D-vs-3D scheduling experiments.

2026-06-11 13:19:51 +00:00

..

add_rmsnorm2d_rdquant

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

batched_contraction

[CK Tile] batched contraction kernel generalizing (#3126)

2025-12-02 13:30:27 +01:00

batched_transpose

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

common

[CK Tile] StreamK support for Bwd Weight grouped convolutions (#5393)

2026-03-27 10:17:10 +01:00

elementwise

[CK_TILE, CK_BUILDER] Add two-stage bwd weight kernels to CK Tile profiler (#5237)

2026-03-12 19:20:15 -06:00

epilogue

[CK_TILE] Flatten nested static_for loops into static_ford (#5939)

2026-04-07 08:36:45 -06:00

flatmm

[CK_TILE] Flatten nested static_for loops into static_ford (#5939)

2026-04-07 08:36:45 -06:00

fmha

[fmha-bwd] Flat cu_id remap for arbitrary CTA_NUM + grid Y/Z override env

2026-06-11 13:19:51 +00:00

fused_moe

[CK] Fix windows build issues (#4819)

2026-02-25 09:12:19 -07:00

gemm

[CK_TILE] Flatten nested static_for loops into static_ford (#5939)

2026-04-07 08:36:45 -06:00

gemm_mx

[CK_TILE ]Revert "[CK_TILE] Enable MXFP6 for MX GEMM op (#5095)" (#5849)

2026-03-27 20:36:39 +00:00

gemm_quant

[CK_TILE] Flatten nested static_for loops into static_ford (#5939)

2026-04-07 08:36:45 -06:00

grouped_convolution

[CK][CK Tile] Conv Bwd Data flush cache and profiling improvements (#6090)

2026-04-04 00:22:22 +00:00

image_to_column

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

layernorm2d

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

norm_reduce

[CK_TILE] Flatten nested static_for loops into static_ford (#5939)

2026-04-07 08:36:45 -06:00

permute

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

pooling

Shuffle fix for gfx950 (#3491)

2026-01-13 09:21:29 -08:00

reduce

[CK_TILE] Flatten nested static_for loops into static_ford (#5939)

2026-04-07 08:36:45 -06:00

rmsnorm2d

Fix redundant cast in model sensitive rmsnorm (#3681)

2026-01-30 10:52:19 +08:00

smoothquant

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

softmax/block

Add padding to cshuffle epilogue to avoid bank conflict (#4274)

2026-02-10 22:52:00 -07:00

sparse_attn

[CK_TILE][FMHA] Add sparse attention VSA (#3341)

2026-01-31 00:59:47 +08:00

topk/block

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

topk_softmax

Shuffle fix for gfx950 (#3491)

2026-01-13 09:21:29 -08:00

add_rmsnorm2d_rdquant.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

batched_contraction.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

batched_transpose.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

common.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

elementwise.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

epilogue.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

flatmm.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

fmha.hpp

[CK_TILE][FMHA] Support microscaling (mxfp8 and mxfp4) on gfx950 (#4368)

2026-03-11 09:59:50 +00:00

fused_moe.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

gemm_mx.hpp

Changed the include order of the new WMMA/MFMA unification framework (#5241)

2026-03-12 09:26:58 +01:00

gemm_quant.hpp

[CK Tile] Eight Waves pipeline GEMM (#4964)

2026-03-16 09:30:54 +01:00

gemm.hpp

[CK Tile] Eight Waves pipeline GEMM (#4964)

2026-03-16 09:30:54 +01:00

grouped_convolution.hpp

Changed the include order of the new WMMA/MFMA unification framework (#5241)

2026-03-12 09:26:58 +01:00

image_to_column.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

layernorm2d.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

moe_flatmm.hpp

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

norm_reduce.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

permute.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

pooling.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

reduce.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

rmsnorm2d.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

smoothquant.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

softmax.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

sparse_attn.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

topk_softmax.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00

topk.hpp

Cleanup and refactoring related to tile loading (#4294)

2026-03-02 12:20:55 +00:00