Illia Silin b8635a25b2 Fix the group of quantization_int8 kernels on MI300. (#695) ...

* replace amd_buffer_atomic_add with hip_atomic_add

* fix grouped_gemm_splitk kernels on mi300

* fix syntax

* revert experimental atomic_add changes

* fix the group of kernels from ticket 723 on MI300

---------

Co-authored-by: Jing Zhang <jizhan@amd.com>

2023-05-03 18:27:04 -05:00

..

impl

Fix the group of quantization_int8 kernels on MI300. (#695)

2023-05-03 18:27:04 -05:00

convolution_backward_data_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

convolution_backward_weight_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

convolution_forward_specialization.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_base.hpp

Remove including of cmath (#551)

2023-01-11 19:52:47 -06:00

device_batched_contraction_multiple_d.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

device_batched_gemm_e_permute.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_batched_gemm_gemm.hpp

Attention with output permutation (#370)

2022-08-23 14:52:56 -05:00

device_batched_gemm_multi_d.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_batched_gemm_multiple_d_gemm_multiple_d.hpp

batched_gemm + multiple_d + gemm + multiple_d (#394)

2022-09-14 17:54:18 -05:00

device_batched_gemm_softmax_gemm_permute.hpp

Add GemmAddSoftmaxGemm support for MSFT ORT (instances and client API) (#576)

2023-02-08 14:34:45 -06:00

device_batched_gemm_softmax_gemm.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_batched_gemm.hpp

Gemm+Bilinear (#316)

2022-07-02 09:15:38 -05:00

device_batchnorm_backward.hpp

BatchNorm backward instance/external API/profiler/tests (#519)

2022-11-30 13:32:20 -06:00

device_batchnorm_forward.hpp

BatchNorm forward instance/external api/profiler/tests/client example (#511)

2022-11-24 18:02:27 -06:00

device_batchnorm_infer.hpp

BatchNorm forward instance/external api/profiler/tests/client example (#511)

2022-11-24 18:02:27 -06:00

device_cgemm.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_contraction_multiple_d.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_conv_bwd_data.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_conv_fwd_bias_activation_add.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_conv_fwd_bias_activation.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_conv_fwd.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_elementwise_normalization.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_elementwise.hpp

Batchnorm inference instances, external API, client examples and gtests (#531)

2023-01-25 17:09:04 -06:00

device_gemm_bias_e_permute.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

device_gemm_multiple_d_layernorm.hpp

Gemm layernorm welford (#413)

2023-01-16 20:08:25 -06:00

device_gemm_multiple_d_multiple_r.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_gemm_multiple_d.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_gemm_reduce.hpp

Improve external interface for GEMM and GEMM+add+add+fastgelu (#311)

2022-06-30 22:11:00 -05:00

device_gemm_splitk.hpp

Improve external interface for GEMM and GEMM+add+add+fastgelu (#311)

2022-06-30 22:11:00 -05:00

device_gemm_xdl_waveletmodel_cshuffle.hpp

Syncing up from internal repo to enable MI300. (#690)

2023-04-28 18:22:59 -05:00

device_gemm.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

device_grouped_contraction_multiple_d.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

device_grouped_conv_bwd_data_multiple_d.hpp

Remove interface 'DeviceGroupedConvBwdData' (#500)

2022-11-09 18:32:17 -06:00

device_grouped_conv_bwd_weight.hpp

Add client example of grouped conv2d backward weight (data type: fp16) (#498)

2022-11-09 18:50:03 -06:00

device_grouped_conv_fwd_dl_multiple_d_nhwc_kyxc_nhwk.hpp

Fix the group of quantization_int8 kernels on MI300. (#695)

2023-05-03 18:27:04 -05:00

device_grouped_conv_fwd_dl_nhwc_kyxc_nhwk.hpp

Update GetTypeString function to generate unique kernel IDs. (#638)

2023-03-15 10:44:42 -05:00

device_grouped_conv_fwd_multiple_d.hpp

Add a denorm test fix (#603)

2023-03-29 15:05:32 -05:00

device_grouped_conv_fwd.hpp

Add client example of grouped conv2d forward (data type: fp16) (#488)

2022-11-09 19:01:58 -06:00

device_grouped_gemm_softmax_gemm_permute_xdl_cshuffle.hpp

Syncing up from internal repo to enable MI300. (#690)

2023-04-28 18:22:59 -05:00

device_grouped_gemm_softmax_gemm_permute.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

device_grouped_gemm_splitk.hpp

Grouped Gemm + SplitK + simplified Kernel Args (#669)

2023-04-24 15:43:36 -05:00

device_grouped_gemm.hpp

Grouped Gemm + SplitK + simplified Kernel Args (#669)

2023-04-24 15:43:36 -05:00

device_multiple_reduce.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_normalization.hpp

Improve normalization (#580)

2023-02-15 11:59:35 -06:00

device_permute.hpp

Remove including of cmath (#551)

2023-01-11 19:52:47 -06:00

device_pool2d_fwd.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

device_reduce.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_softmax.hpp

Use double for all scaling values and float-point constant values at the Device Op API (#557)

2023-01-18 12:02:50 -06:00

device_splitk_contraction_multiple_d_xdl_cshuffle.hpp

Syncing up from internal repo to enable MI300. (#690)

2023-04-28 18:22:59 -05:00

device_splitk_contraction_multiple_d.hpp

Example contraction splitk (#430)

2022-10-11 17:54:34 -05:00

gemm_specialization.hpp

Implement padding and sanity checks for fused GEMM+GEMM (#376)

2022-08-23 10:01:02 -05:00

masking_specialization.hpp

Input/output permutation for fused attention (#460)

2022-10-27 14:58:20 -06:00

matrix_padder.hpp

batched_gemm + multiple_d + gemm + multiple_d (#394)

2022-09-14 17:54:18 -05:00

reduction_operator_mapping.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

tensor_layout.hpp

Conv bwd data multiple d (#404)

2022-09-19 11:25:28 -05:00

tensor_specialization.hpp

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

welford_helper.hpp

Batchnorm-forward implemented using welford method to calculate variance (#403)

2022-10-27 18:52:54 -06:00