Zoltán Lakatos 301eb5cf08 Implement device grouped gemm fixed nk multi abd for rdna4 (#3619) ...

* device struct implementation

* added xdl grouped multi abd fixed nk testing

* wmma implementation fixed

* avoid unnecessary device mem allocation and code cleanups

* cleanup instances definitions

* wmma examples added

* code cleanups

* fix clang format

* typo and compilation fixes related to reference gemm

* fix compilation error due to std::remove_cvref_t

* added missing hip_check_error includes

* correction to example instances

* review commentes addressed

* removed split-k from testing

* code formatting

---------

Co-authored-by: Zoltán Lakatos <zoltan.lakatos@streamhpc.com>
Co-authored-by: illsilin_amdeng <Illia.Silin@amd.com>

2026-02-02 13:58:11 -08:00

..

impl

Implement device grouped gemm fixed nk multi abd for rdna4 (#3619)

2026-02-02 13:58:11 -08:00

conv_tensor_rearrange_op.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

convolution_backward_data_specialization.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

convolution_backward_weight_specialization.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

convolution_forward_specialization.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_avgpool_bwd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_base.hpp

Improve XDL to WMMA porting for grouped conv fwd (#3456)

2025-12-19 15:58:51 -07:00

device_batched_contraction_multiple_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batched_gemm_e_permute.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batched_gemm_gemm.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batched_gemm_multi_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batched_gemm_multiple_d_gemm_multiple_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batched_gemm_softmax_gemm_permute.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batched_gemm_softmax_gemm.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batched_gemm.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batchnorm_backward.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batchnorm_forward.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_batchnorm_infer.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_cgemm.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_contraction_multiple_abd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_contraction_multiple_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_conv_bwd_data.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_conv_fwd_bias_activation_add.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_conv_fwd_bias_activation.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_conv_fwd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_conv_tensor_rearrange.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_elementwise_normalization.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_elementwise_scale.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_elementwise.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_bias_e_permute.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_dequantB.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_multiple_abd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_multiple_d_ab_scale.hpp

Wmma support for gemm_ab_scale (#3314)

2025-12-11 09:06:20 +01:00

device_gemm_multiple_d_layernorm.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_multiple_d_multiple_r.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_multiple_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_mx.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_reduce.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_splitk.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_streamk_v2.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_streamk.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm_v2.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_gemm.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_contraction_multiple_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_conv_bwd_data_multiple_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_conv_bwd_weight_multiple_d.hpp

[Conv] Enable bwd weight splitk autodeduction with cap (#3656)

2026-01-29 17:40:28 +00:00

device_grouped_conv_bwd_weight.hpp

[Conv] Enable bwd weight splitk autodeduction with cap (#3656)

2026-01-29 17:40:28 +00:00

device_grouped_conv_fwd_multiple_abd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_conv_fwd_multiple_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_conv_fwd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_gemm_fixed_nk.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_gemm_multi_abd_fixed_nk.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_gemm_multi_abd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_gemm_softmax_gemm_permute.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_gemm_splitk.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_grouped_gemm_tile_loop.hpp

Implement grouped gemm tile loop for RDNA4 (#3304)

2026-01-13 07:14:23 +01:00

device_grouped_gemm.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_max_pool_bwd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_multiple_reduce.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_normalization_bwd_data.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_normalization_bwd_gamma_beta.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_normalization_fwd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_permute.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_pool_fwd.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_put_element.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_reduce_multi_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_reduce.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_softmax.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

device_splitk_contraction_multiple_d.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

gemm_specialization.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

helper.hpp

chore(copyright): update copyright header for include directory (#3224)

2025-11-18 10:17:18 -08:00

masking_specialization.hpp

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

matrix_padder.hpp

Implement batched gemm add relu gemm add for rdna4 (#3391)

2026-01-20 13:06:59 -08:00

reduction_operator_mapping.hpp

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

tensor_layout.hpp

[Compiler] Addressing new compiler warnings (#3640)

2026-02-02 09:39:48 -08:00

tensor_specialization.hpp

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00

welford_helper.hpp

chore(copyright): update copyright header for include directory (#3293)

2025-11-26 11:00:05 -07:00