Haocong WANG 789c15d703 [Navi3x] Add Device Operations (#567) ...

* wmma_op + unit test

* add arch limitation to wmma test

* change arch limitation

* Refactor + Add all type unit test(int4 compile failed)

* Add f32_16x16x16_bf16 unit test

* tempsave

* tempsave

* tempsave

* runtime bug, cannot find symbol

* workaround for incorrect HIP warpSize return value

* debugging

* tempsave

* Correctness OK, waiting for optimization

* Tidy up + format

* temp save

* temp save, reproduce the v_bfi_b32 issue

* add inline asm for wmmaop test

* tidy up

* clean some debug purpose code

* discard some codes

* clang format

* clang format

* compiler issue fixed + increase tile size

* navi3x_multipleD+example

* temp save

* workable

* batchedgemm[OK], groupconv[debug]

* groupconv: Sanity check[OK], Performance[Bad]

* navi3x_groupconv_need_optimization

* format

* Add arch limitation to all wmma examples

* fix bug: example30 input conv args

[ROCm/composable_kernel commit: 0cfda84d05]

2023-02-15 11:50:51 -06:00

..

impl

[Navi3x] Add Device Operations (#567)

2023-02-15 11:50:51 -06:00

convolution_backward_data_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

convolution_backward_weight_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

convolution_forward_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_base.hpp

Remove including of cmath (#551)

2023-01-11 19:52:47 -06:00

device_batched_contraction_multiple_d.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

device_batched_gemm_e_permute.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_batched_gemm_gemm.hpp

Attention with output permutation (#370)

2022-08-23 14:52:56 -05:00

device_batched_gemm_multi_d.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_batched_gemm_multiple_d_gemm_multiple_d.hpp

batched_gemm + multiple_d + gemm + multiple_d (#394)

2022-09-14 17:54:18 -05:00

device_batched_gemm_softmax_gemm_permute.hpp

Add GemmAddSoftmaxGemm support for MSFT ORT (instances and client API) (#576)

2023-02-08 14:34:45 -06:00

device_batched_gemm_softmax_gemm.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_batched_gemm.hpp

Gemm+Bilinear (#316)

2022-07-02 09:15:38 -05:00

device_batchnorm_backward.hpp

BatchNorm backward instance/external API/profiler/tests (#519)

2022-11-30 13:32:20 -06:00

device_batchnorm_forward.hpp

BatchNorm forward instance/external api/profiler/tests/client example (#511)

2022-11-24 18:02:27 -06:00

device_batchnorm_infer.hpp

BatchNorm forward instance/external api/profiler/tests/client example (#511)

2022-11-24 18:02:27 -06:00

device_cgemm.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_contraction_multiple_d.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_conv_bwd_data.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_conv_fwd_bias_activation_add.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_conv_fwd_bias_activation.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_conv_fwd.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_elementwise_normalization.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_elementwise.hpp

Batchnorm inference instances, external API, client examples and gtests (#531)

2023-01-25 17:09:04 -06:00

device_gemm_bias_e_permute.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_gemm_multiple_d_layernorm.hpp

Gemm layernorm welford (#413)

2023-01-16 20:08:25 -06:00

device_gemm_multiple_d_multiple_r.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_gemm_multiple_d.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_gemm_reduce.hpp

Improve external interface for GEMM and GEMM+add+add+fastgelu (#311)

2022-06-30 22:11:00 -05:00

device_gemm_splitk.hpp

Improve external interface for GEMM and GEMM+add+add+fastgelu (#311)

2022-06-30 22:11:00 -05:00

device_gemm_xdl_waveletmodel_cshuffle.hpp

Wavelet (inter-wave consumer-producer) GEMM (#310)

2023-01-18 12:00:02 -06:00

device_gemm.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_grouped_contraction_multiple_d.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

device_grouped_conv_bwd_data_multiple_d.hpp

Remove interface 'DeviceGroupedConvBwdData' (#500)

2022-11-09 18:32:17 -06:00

device_grouped_conv_bwd_weight.hpp

Add client example of grouped conv2d backward weight (data type: fp16) (#498)

2022-11-09 18:50:03 -06:00

device_grouped_conv_fwd_dl_multiple_d_nhwc_kyxc_nhwk.hpp

Add multiple d gridwise gemm on Navi21 for ResNet50 (#517)

2022-12-02 11:42:31 -06:00

device_grouped_conv_fwd_dl_nhwc_kyxc_nhwk.hpp

Add Conv Forward on Navi21 for ResNet50 (#490)

2022-10-31 10:24:25 -06:00

device_grouped_conv_fwd_multiple_d.hpp

Conv bwd data multiple d (#404)

2022-09-19 11:25:28 -05:00

device_grouped_conv_fwd.hpp

Add client example of grouped conv2d forward (data type: fp16) (#488)

2022-11-09 19:01:58 -06:00

device_grouped_gemm_softmax_gemm_permute_xdl_cshuffle.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_grouped_gemm_softmax_gemm_permute.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_grouped_gemm.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_multiple_reduce.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_normalization.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_permute.hpp

Remove including of cmath (#551)

2023-01-11 19:52:47 -06:00

device_pool2d_fwd.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_reduce.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_softmax.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_splitk_contraction_multiple_d_xdl_cshuffle.hpp

Example contraction splitk (#430)

2022-10-11 17:54:34 -05:00

device_splitk_contraction_multiple_d.hpp

Example contraction splitk (#430)

2022-10-11 17:54:34 -05:00

gemm_specialization.hpp

Implement padding and sanity checks for fused GEMM+GEMM (#376)

2022-08-23 10:01:02 -05:00

masking_specialization.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

matrix_padder.hpp

batched_gemm + multiple_d + gemm + multiple_d (#394)

2022-09-14 17:54:18 -05:00

reduction_operator_mapping.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

tensor_layout.hpp

Conv bwd data multiple d (#404)

2022-09-19 11:25:28 -05:00

tensor_specialization.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

welford_helper.hpp

Batchnorm-forward implemented using welford method to calculate variance (#403)

2022-10-27 18:52:54 -06:00