Raman R jana 1cfa87608a Wavelet (inter-wave consumer-producer) GEMM (#310) ...

* wavelet gemm programming model support for CK

* GEMM pipeline update for wavelet progrmmaing model

* Updated wavelet programming pipeline

* fixes for global-write for math-wave

* fixed bug in global writes

* Updated comments for better readability

* fixed clang format errors

* added block_lds without barrier sync

* clean

* clean

* clean

* clean

* refactor

* prototype

4 layouts

fix default stride

all problem sizes

tidy

move file

update build script

restore old file

fix build

* refactor standalone test to use gemm test harness

* simplify gemm test

* update build script

* remove redundant

* early return when cmd arg doesn't match

* tidy

* report failure when result not validated

* tidy

* Add comment depicting B2C mapping pattern.

* Formatting & comments.

* Comparison with custom B2C mapping pattern.

* Example for wavelet gemm.

* Add wavelet to Gemm standalone test.

* Remove debug code.

* Remove dangling #endif directive.

Co-authored-by: root <Raman Jana>
Co-authored-by: Chao Liu <chao.liu2@amd.com>
Co-authored-by: Adam Osewski <aosewski@amd.com>
Co-authored-by: Anthony Chang <ac.chang@outlook.com>
Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>

2023-01-18 12:00:02 -06:00

..

impl

Wavelet (inter-wave consumer-producer) GEMM (#310)

2023-01-18 12:00:02 -06:00

convolution_backward_data_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

convolution_backward_weight_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

convolution_forward_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_base.hpp

Remove including of cmath (#551)

2023-01-11 19:52:47 -06:00

device_batched_contraction_multiple_d.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

device_batched_gemm_e_permute.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_batched_gemm_gemm.hpp

Attention with output permutation (#370)

2022-08-23 14:52:56 -05:00

device_batched_gemm_multi_d.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_batched_gemm_multiple_d_gemm_multiple_d.hpp

batched_gemm + multiple_d + gemm + multiple_d (#394)

2022-09-14 17:54:18 -05:00

device_batched_gemm_softmax_gemm_permute.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_batched_gemm_softmax_gemm.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_batched_gemm.hpp

Gemm+Bilinear (#316)

2022-07-02 09:15:38 -05:00

device_batchnorm_backward.hpp

BatchNorm backward instance/external API/profiler/tests (#519)

2022-11-30 13:32:20 -06:00

device_batchnorm_forward.hpp

BatchNorm forward instance/external api/profiler/tests/client example (#511)

2022-11-24 18:02:27 -06:00

device_batchnorm_infer.hpp

BatchNorm forward instance/external api/profiler/tests/client example (#511)

2022-11-24 18:02:27 -06:00

device_cgemm.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_contraction_multiple_d.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_conv_bwd_data.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_conv_fwd_bias_activation_add.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_conv_fwd_bias_activation.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_conv_fwd.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_elementwise_2d.hpp

Gridwise elementwise 2d (#466)

2022-12-12 09:18:10 -06:00

device_elementwise_base.hpp

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

device_elementwise_normalization.hpp

Fused elementwise normalization (#492)

2022-11-03 12:01:58 -06:00

device_gemm_bias_e_permute.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_gemm_multiple_d_layernorm.hpp

Gemm layernorm welford (#413)

2023-01-16 20:08:25 -06:00

device_gemm_multiple_d_multiple_r.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_gemm_multiple_d.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_gemm_reduce.hpp

Improve external interface for GEMM and GEMM+add+add+fastgelu (#311)

2022-06-30 22:11:00 -05:00

device_gemm_splitk.hpp

Improve external interface for GEMM and GEMM+add+add+fastgelu (#311)

2022-06-30 22:11:00 -05:00

device_gemm_xdl_waveletmodel_cshuffle.hpp

Wavelet (inter-wave consumer-producer) GEMM (#310)

2023-01-18 12:00:02 -06:00

device_gemm.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_grouped_contraction_multiple_d.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

device_grouped_conv_bwd_data_multiple_d.hpp

Remove interface 'DeviceGroupedConvBwdData' (#500)

2022-11-09 18:32:17 -06:00

device_grouped_conv_bwd_weight.hpp

Add client example of grouped conv2d backward weight (data type: fp16) (#498)

2022-11-09 18:50:03 -06:00

device_grouped_conv_fwd_dl_multiple_d_nhwc_kyxc_nhwk.hpp

Add multiple d gridwise gemm on Navi21 for ResNet50 (#517)

2022-12-02 11:42:31 -06:00

device_grouped_conv_fwd_dl_nhwc_kyxc_nhwk.hpp

Add Conv Forward on Navi21 for ResNet50 (#490)

2022-10-31 10:24:25 -06:00

device_grouped_conv_fwd_multiple_d.hpp

Conv bwd data multiple d (#404)

2022-09-19 11:25:28 -05:00

device_grouped_conv_fwd.hpp

Add client example of grouped conv2d forward (data type: fp16) (#488)

2022-11-09 19:01:58 -06:00

device_grouped_gemm_softmax_gemm_permute_xdl_cshuffle.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_grouped_gemm_softmax_gemm_permute.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_grouped_gemm.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_multiple_reduce.hpp

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

device_normalization.hpp

Refine layernorm naming and test code (#497)

2022-11-02 16:57:28 -06:00

device_permute.hpp

Remove including of cmath (#551)

2023-01-11 19:52:47 -06:00

device_pool2d_fwd.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_reduce.hpp

Reduction external API and client examples (#493)

2023-01-16 22:18:06 -06:00

device_softmax.hpp

Softmax unit-test reduction across all and non innermost dims cases. (#406)

2022-11-02 16:46:08 -06:00

device_splitk_contraction_multiple_d_xdl_cshuffle.hpp

Example contraction splitk (#430)

2022-10-11 17:54:34 -05:00

device_splitk_contraction_multiple_d.hpp

Example contraction splitk (#430)

2022-10-11 17:54:34 -05:00

gemm_specialization.hpp

Implement padding and sanity checks for fused GEMM+GEMM (#376)

2022-08-23 10:01:02 -05:00

masking_specialization.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

matrix_padder.hpp

batched_gemm + multiple_d + gemm + multiple_d (#394)

2022-09-14 17:54:18 -05:00

reduction_operator_mapping.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

tensor_layout.hpp

Conv bwd data multiple d (#404)

2022-09-19 11:25:28 -05:00

tensor_specialization.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

welford_helper.hpp

Batchnorm-forward implemented using welford method to calculate variance (#403)

2022-10-27 18:52:54 -06:00