zjing14 602c4cc0d9 Optimizing fp8_fp16 mixedprec gemm (#1150) ...

* add delayed cvt

* extend fp16 gemm_splitk instances for fp8_fp16 gemm

* add f8 example

* add 128 kperblk instances for fp8

* add kpb128 instance

* added more instances into kpb128

* clean code

* clean code

* fix

* fix

* fixed

* Update example/35_splitK_gemm/splitK_gemm_xdl_fp16_fp8.cpp

Co-authored-by: Bartłomiej Kocot <barkocot@amd.com>

* Update include/ck/tensor_operation/gpu/thread/threadwise_tensor_slice_transfer.hpp

Co-authored-by: Bartłomiej Kocot <barkocot@amd.com>

* Update library/src/tensor_operation_instance/gpu/gemm_splitk/device_gemm_xdl_splitk_f16_fp8_f16_mk_nk_mn_kpb128_instance.cpp

Co-authored-by: Bartłomiej Kocot <barkocot@amd.com>

---------

Co-authored-by: Jing Zhang <jizha@amd.com>
Co-authored-by: Bartłomiej Kocot <barkocot@amd.com>

2024-02-12 09:45:42 -08:00

..

batchnorm_multiblock

[HotFix] add config and version files to pass on build info (#856)

2023-08-23 11:36:17 -07:00

gemm_layernorm

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

normalization

layernorm and groupnorm backward data (#1083)

2023-12-19 04:23:11 +08:00

block_to_ctile_map.hpp

[GEMM] Optimization for MI200/300. (#1135)

2024-01-19 07:02:22 -06:00

gridwise_2d_multiple_reduction_multiblock.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_2d_multiple_reduction_threadwise.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_2d_reduction_multiblock.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_2d_reduction_threadwise.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_batched_gemm_gemm_xdl_cshuffle_v1.hpp

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

gridwise_batched_gemm_multiple_d_gemm_multiple_d_xdl_cshuffle_v1.hpp

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

gridwise_batched_gemm_multiple_d_softmax_gemm_xdl_cshuffle_v1.hpp

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

gridwise_batched_gemm_softmax_gemm_xdl_cshuffle_v1.hpp

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

gridwise_batchnorm_backward_blockwise_welford.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_batchnorm_forward_blockwise_welford.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_elementwise_1d_scale.hpp

Hip tensor permute (#1002)

2023-11-13 11:15:48 -06:00

gridwise_elementwise_1d.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_elementwise_2d.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_elementwise_3d.hpp

Transpose 3d (#984)

2023-11-08 19:45:07 -06:00

gridwise_elementwise_layernorm_welford_variance.hpp

enable compilation of INSTANCES_ONLY for Windows (#1082)

2023-12-20 14:34:53 -08:00

gridwise_gemm_bias_add_reduce_xdl_cshuffle_v1.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_dl_multiple_d.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_gemm_dl_v1r3.hpp

Redesign the DPP8 GEMM kernel to use warp-wise component (#863)

2023-09-06 11:44:09 -05:00

gridwise_gemm_dpp.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_multiple_abd_xdl_cshuffle.hpp

Support multi AB for grouped conv fwd xdl (#1027)

2023-11-10 15:54:44 +01:00

gridwise_gemm_multiple_d_multiple_r_xdl_cshuffle.hpp

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

gridwise_gemm_multiple_d_wmma_cshuffle.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_multiple_d_xdl_cshuffle_lds_direct_load.hpp

Implement direct loads split-K GEMM kernel (#1137)

2024-02-07 01:08:34 +01:00

gridwise_gemm_multiple_d_xdl_cshuffle.hpp

Grouped conv bwd data with fp16 input and bf8fp8 comp (#962)

2023-10-04 18:04:27 -05:00

gridwise_gemm_multiple_d_xdl_splitk_cshuffle.hpp

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

gridwise_gemm_pipeline_selector.hpp

Add basic support for direct loads from global to LDS (#999)

2023-11-25 13:35:22 +01:00

gridwise_gemm_pipeline_v1.hpp

Redesign the DPP8 GEMM kernel to use warp-wise component (#863)

2023-09-06 11:44:09 -05:00

gridwise_gemm_pipeline_v2.hpp

Fixed Gemmv2r3 kpad (#938)

2023-09-26 18:40:00 -05:00

gridwise_gemm_pipeline_v3.hpp

initial stream-k implementation with example (#699)

2023-07-26 14:18:15 -05:00

gridwise_gemm_pipeline_v4_direct_load.hpp

Add support for double buffering in direct load GEMM kernel (#1052)

2023-12-03 23:08:47 +01:00

gridwise_gemm_reduce_xdl_cshuffle_v1.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_split_k_multiple_d_xdl_cshuffle_v2.hpp

Fp16/fp8 mixed-precision Gemm with multiply+add fusion (#865)

2023-08-28 16:27:32 -05:00

gridwise_gemm_split_k_multiple_d_xdl_cshuffle.hpp

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

gridwise_gemm_waveletmodel.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_gemm_wmma.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdl_cshuffle_v1.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdl_cshuffle_v2.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdl_layernorm_cshuffle_v1.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdl_waveletmodel_cshuffle.hpp

Add fp8 @ bf8 gemm support and example (#933)

2023-10-02 16:39:03 -05:00

gridwise_gemm_xdlops_bwd_weight.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdlops_skip_b_lds_v1.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdlops_splitk_lds_direct_load.hpp

Implement direct loads split-K GEMM kernel (#1137)

2024-02-07 01:08:34 +01:00

gridwise_gemm_xdlops_streamk.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdlops_v2r3.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdlops_v2r4.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdlops_v2r4r2.hpp

Optimizing fp8_fp16 mixedprec gemm (#1150)

2024-02-12 09:45:42 -08:00

gridwise_gemm_xdlops_v3r1.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdlops_v3r2.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_gemm_xdlops_v3r3.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00

gridwise_permute.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_put_element_1d.hpp

Average pool backward deviceOP and example (#797)

2023-08-10 12:04:35 +08:00

gridwise_set_buffer_value.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_set_multiple_buffer_value.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_softmax.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

gridwise_sparse_embeddings_forward_layernorm.hpp

[HotFix] add config and version files to pass on build info (#856)

2023-08-23 11:36:17 -07:00

gridwise_tensor_rearrange.hpp

Add support for more Navi2x and Navi3x models. (#1152)

2024-02-02 11:35:26 -08:00