Andriy Roshchenko 79b0bfeb41 MX GEMM - Add FP8 GEMM Tests for Different Layouts (#2152) ...

* Add gemm_mx_fp8_bf8 example with row-major B

* Add more overloads of MX MFMA instructions

* Add MK_KN (RRR) tests

* Add KM_NK (CCR) tests

* Add more problem sizes to Large tests

* Add test_gemm_mx to the list of regression tests

2025-05-01 11:55:48 -06:00

..

batchnorm_multiblock

…

gemm_layernorm

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

normalization

…

block_to_ctile_map.hpp

MI308 fix for streamk 1-Tile floating point exception (#2101)

2025-04-21 11:44:07 -07:00

gridwise_2d_multiple_reduction_multiblock.hpp

…

gridwise_2d_multiple_reduction_threadwise.hpp

…

gridwise_2d_reduction_multiblock.hpp

…

gridwise_2d_reduction_threadwise_multi_d.hpp

…

gridwise_2d_reduction_threadwise.hpp

…

gridwise_batched_gemm_gemm_xdl_cshuffle_v1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_batched_gemm_multiple_d_gemm_multiple_d_xdl_cshuffle_v1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_batched_gemm_multiple_d_softmax_gemm_xdl_cshuffle_v1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_batched_gemm_softmax_gemm_wmma_cshuffle.hpp

…

gridwise_batched_gemm_softmax_gemm_xdl_cshuffle_v1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_batchnorm_backward_blockwise_welford.hpp

…

gridwise_batchnorm_forward_blockwise_welford.hpp

…

gridwise_elementwise_1d_scale.hpp

…

gridwise_elementwise_2d.hpp

Grouped conv backward data GKCYX support (#2029)

2025-04-01 13:24:38 -07:00

gridwise_elementwise_layernorm_welford_variance.hpp

…

gridwise_fpAintB_gemm_wmma.hpp

…

gridwise_gemm_bias_add_reduce_xdl_cshuffle_v1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_dl_multiple_d.hpp

…

gridwise_gemm_dl_v1r3.hpp

…

gridwise_gemm_dpp.hpp

…

gridwise_gemm_multiple_abd_xdl_cshuffle.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_multiple_d_multiple_r_xdl_cshuffle.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_multiple_d_wmma_cshuffle.hpp

…

gridwise_gemm_multiple_d_xdl_cshuffle_lds_direct_load.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_multiple_d_xdl_cshuffle.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_multiple_d_xdl_splitk_cshuffle.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_pipeline_selector.hpp

Rebase the PR #1520 to ROCm repo. (#1574)

2025-02-20 18:58:14 -08:00

gridwise_gemm_pipeline_v1.hpp

…

gridwise_gemm_pipeline_v2.hpp

…

gridwise_gemm_pipeline_v3.hpp

…

gridwise_gemm_pipeline_v4_direct_load.hpp

…

gridwise_gemm_reduce_xdl_cshuffle_v1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_split_k_multiple_d_xdl_cshuffle_v2.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_split_k_multiple_d_xdl_cshuffle.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_waveletmodel.hpp

…

gridwise_gemm_wmma_cshuffle_v3.hpp

DeviceGemm_Wmma_CShuffleV3 with BlockGemmPipelineVersion::v3 (#2096)

2025-04-28 10:14:21 +05:00

gridwise_gemm_wmma.hpp

…

gridwise_gemm_xdl_cshuffle_conv_v3.hpp

Integrate universal gemm with conv bwd data and add SplitK (#1315)

2025-04-28 23:54:49 +02:00

gridwise_gemm_xdl_cshuffle_streamk_v3.hpp

Split env.hpp header from the ck.hpp header. (#2049)

2025-04-03 15:30:21 -07:00

gridwise_gemm_xdl_cshuffle_v1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_xdl_cshuffle_v2.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_xdl_cshuffle_v3_b_preshuffle.hpp

Split env.hpp header from the ck.hpp header. (#2049)

2025-04-03 15:30:21 -07:00

gridwise_gemm_xdl_cshuffle_v3_b_scale.hpp

Split env.hpp header from the ck.hpp header. (#2049)

2025-04-03 15:30:21 -07:00

gridwise_gemm_xdl_cshuffle_v3_multi_abd.hpp

Split env.hpp header from the ck.hpp header. (#2049)

2025-04-03 15:30:21 -07:00

gridwise_gemm_xdl_cshuffle_v3_multi_d_ab_scale.hpp

[Block Scale GEMM] Optimized block scale gemm (#1950)

2025-03-11 10:11:21 -07:00

gridwise_gemm_xdl_cshuffle_v3_multi_d_b_preshuffle.hpp

[A8W8 GEMM] Optimized weight-preshuffled implementation & add quantization datatype for CK TILE rms_norm (#1862)

2025-02-20 14:00:27 -08:00

gridwise_gemm_xdl_cshuffle_v3_multi_d.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_xdl_cshuffle_v3_mx.hpp

MX GEMM - Add FP8 GEMM Tests for Different Layouts (#2152)

2025-05-01 11:55:48 -06:00

gridwise_gemm_xdl_cshuffle_v3.hpp

Split env.hpp header from the ck.hpp header. (#2049)

2025-04-03 15:30:21 -07:00

gridwise_gemm_xdl_layernorm_cshuffle_v1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_xdl_waveletmodel_cshuffle.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_xdlops_bwd_weight.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_xdlops_skip_b_lds_v1.hpp

…

gridwise_gemm_xdlops_splitk_lds_direct_load.hpp

…

gridwise_gemm_xdlops_streamk.hpp

…

gridwise_gemm_xdlops_v2r3.hpp

Change flag to CK_GFX90A_DENORM_WORKAROUND (#1817)

2025-01-28 09:58:39 -05:00

gridwise_gemm_xdlops_v2r4.hpp

…

gridwise_gemm_xdlops_v2r4r2.hpp

Split env.hpp header from the ck.hpp header. (#2049)

2025-04-03 15:30:21 -07:00

gridwise_gemm_xdlops_v3r1.hpp

Remove CK_USE_AMD_MFMA_GFX950 (#1935)

2025-03-04 10:32:25 -08:00

gridwise_gemm_xdlops_v3r2.hpp

…

gridwise_gemm_xdlops_v3r3.hpp

…

gridwise_moe_gemm.hpp

Moe gemm activation (#2026)

2025-04-23 10:35:34 +08:00

gridwise_permute.hpp

…

gridwise_put_element_1d.hpp

…

gridwise_set_buffer_value.hpp

…

gridwise_set_multiple_buffer_value.hpp

…

gridwise_softmax.hpp

…

gridwise_sparse_embeddings_forward_layernorm_builtins.hpp

Replace buffer load/store intrinsics with builtins (#1876)

2025-03-05 14:33:28 -08:00

gridwise_sparse_embeddings_forward_layernorm.hpp

…

gridwise_tensor_rearrange.hpp

…