Mingtao Gu d9f1ead347 Added Int4 mixed batch gemm support (#1839) ...

* remove redundant kernels.

* added batched_gemm_xdl_fp16int4_b_scale_v3

* Enabled the split K.

* added the batched_gemm_b_scale ckProfiler, meet function issue

* fix some typo

* fix ckProfiler build issue

* fix some bugs

* updated some debug info

* comment some code

* Fix

* fixed some bugs and refactor the code

* fixed a function bug.

* formatted files.

* formatted

* uncommented the ckProfiler CMakeLists

* fixed.

* fix ckProfiler for batched_gemm_b_scale

---------

Co-authored-by: mtgu0705 <mtgu@amd.com>
Co-authored-by: aska-0096 <haocwang@amd.com>
Co-authored-by: Bartlomiej Kocot <barkocot@amd.com>

2025-02-10 11:17:02 +08:00

..

CMakeLists.txt

Added Int4 mixed batch gemm support (#1839)

2025-02-10 11:17:02 +08:00

profile_avg_pool2d_bwd.cpp

Pool2d max/avg kernel in the BWD version (#1494)

2024-09-12 11:47:52 +02:00

profile_avg_pool3d_bwd.cpp

MaxPool & AvgPool bwd instances, test, ckProfiler, client example (#861)

2023-08-31 21:01:50 +08:00

profile_batched_gemm_add_relu_gemm_add.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_batched_gemm_b_scale.cpp

Added Int4 mixed batch gemm support (#1839)

2025-02-10 11:17:02 +08:00

profile_batched_gemm_gemm.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_batched_gemm_multi_d.cpp

[HotFix] add config and version files to pass on build info (#856)

2023-08-23 11:36:17 -07:00

profile_batched_gemm_reduce.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_batched_gemm.cpp

Add DeviceBatchedGemmMultipleD_Dl (#732)

2023-06-12 08:37:15 -05:00

profile_batchnorm_bwd.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_batchnorm_fwd.cpp

Batchnorm splitk single kernel (#771)

2023-07-06 10:58:55 -05:00

profile_batchnorm_infer.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_contraction_bilinear.cpp

Extend support for contraction 6D (#1207)

2024-04-09 23:46:21 +02:00

profile_contraction_scale.cpp

Extend support for contraction 6D (#1207)

2024-04-09 23:46:21 +02:00

profile_conv_bwd_data.cpp

[HotFix] add config and version files to pass on build info (#856)

2023-08-23 11:36:17 -07:00

profile_conv_fwd_bias_relu_add.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_conv_fwd_bias_relu.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_conv_fwd.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_conv_tensor_rearrange.cpp

Add support for groups in Img2Col/Col2Img (#1007)

2023-10-31 10:46:32 +01:00

profile_gemm_ab_scale.cpp

[GEMM] F8 GEMM, performance optimized. (#1384)

2024-07-19 22:06:52 +08:00

profile_gemm_add_add_fastgelu.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_gemm_add_fastgelu.cpp

Add support for mixed-precision f16bf16_int8 gemm (#1127)

2024-02-07 15:54:13 +01:00

profile_gemm_add_multiply.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_gemm_add_relu_add_layernorm.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_gemm_add_relu.cpp

Add support for mixed-precision f16bf16_int8 gemm (#1127)

2024-02-07 15:54:13 +01:00

profile_gemm_add_silu.cpp

Add support for mixed-precision f16bf16_int8 gemm (#1127)

2024-02-07 15:54:13 +01:00

profile_gemm_add.cpp

Add support for mixed-precision f16bf16_int8 gemm (#1127)

2024-02-07 15:54:13 +01:00

profile_gemm_b_scale.cpp

Implement the fp16xint4 scale weight only kernel for Ali (#1786)

2025-01-03 18:35:21 +08:00

profile_gemm_bias_add_reduce.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_gemm_bilinear.cpp

[Navi3x] Add fp16/int8 wmma conv forward instances (#746)

2023-09-07 21:59:26 -05:00

profile_gemm_fastgelu.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_gemm_multiply_add.cpp

Refactor f8_t, add bf8_t (#792)

2023-09-12 17:04:27 -05:00

profile_gemm_multiply_multiply.cpp

Dev/merge u8w8 (#1774)

2025-01-13 10:25:14 -08:00

profile_gemm_reduce.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_gemm_splitk.cpp

Add an option to change the number of warm-up cycles and iterations. (#1124)

2024-01-09 09:43:08 -08:00

profile_gemm_streamk.cpp

initial stream-k implementation with example (#699)

2023-07-26 14:18:15 -05:00

profile_gemm_universal_batched.cpp

Add SplitK support into Batched GEMM V3 (#1729)

2024-12-13 21:08:35 +01:00

profile_gemm_universal_reduce.cpp

Universal gemm splitk using reduce (with multi-d) (#1341)

2024-07-19 22:01:22 +08:00

profile_gemm_universal_streamk.cpp

BF16 GEMM Stream-K (#1541)

2025-01-02 10:30:04 -08:00

profile_gemm_universal.cpp

Jing's contribution: prototype of mixed precision gemm FP16/BF16xint4 GEMM (#1762)

2025-01-02 11:48:06 +08:00

profile_gemm.cpp

Fixing most of the cppcheck errors. (#1142)

2024-01-24 13:47:48 -08:00

profile_grouped_conv_bwd_data.cpp

Grouped 3d conv backward data support (#799)

2023-07-18 11:01:33 -05:00

profile_grouped_conv_bwd_weight.cpp

[POST MERGE PR] Enable grouped conv bwd wei bf16 NGCHW (#1594)

2024-10-23 12:02:33 +02:00

profile_grouped_conv_fwd_outelementop.cpp

Add ckProfiler support for forward 3D convolutions with OUT element-wise operations. (#1354)

2024-07-08 10:55:54 -07:00

profile_grouped_conv_fwd.cpp

Add support for NGCHW in grouped conv fwd (#1499)

2024-09-20 10:45:46 +02:00

profile_grouped_gemm_fastgelu.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_grouped_gemm_fixed_nk.cpp

Added bf16 instances grouped gemm fixed nk (#1825)

2025-01-20 09:13:09 -08:00

profile_grouped_gemm_multiply_tile_loop.cpp

Switch to universal gemm in grouped gemm tile loop (#1335)

2024-06-18 09:01:49 -05:00

profile_grouped_gemm_tile_loop.cpp

Grouped GEMM Multiple D tile loop. (#1247)

2024-04-25 15:12:53 -05:00

profile_grouped_gemm.cpp

Polished Grouped GEMM APIs and new BF16 instances (#1600)

2024-11-27 13:02:44 +01:00

profile_groupnorm_bwd_data.cpp

layernorm and groupnorm backward data (#1083)

2023-12-19 04:23:11 +08:00

profile_groupnorm_bwd_gamma_beta.cpp

layernorm & groupnorm bwd gamma beta (#1133)

2024-01-25 19:53:15 +08:00

profile_groupnorm_fwd.cpp

layernorm and groupnorm backward data (#1083)

2023-12-19 04:23:11 +08:00

profile_layernorm_bwd_data.cpp

layernorm and groupnorm backward data (#1083)

2023-12-19 04:23:11 +08:00

profile_layernorm_bwd_gamma_beta.cpp

layernorm & groupnorm bwd gamma beta (#1133)

2024-01-25 19:53:15 +08:00

profile_layernorm_fwd.cpp

Fix F16 type (#1583)

2024-11-06 11:32:44 -08:00

profile_max_pool2d_bwd.cpp

Pool2d max/avg kernel in the BWD version (#1494)

2024-09-12 11:47:52 +02:00

profile_max_pool2d_fwd.cpp

Add pool2d int8 and fp8 instances (#1508)

2024-09-13 10:18:21 -07:00

profile_max_pool3d_bwd.cpp

MaxPool & AvgPool bwd instances, test, ckProfiler, client example (#861)

2023-08-31 21:01:50 +08:00

profile_permute_scale.cpp

Add elementwise with dynamic vector dim (#1198)

2024-03-22 10:40:43 +01:00

profile_pool3d_fwd.cpp

Extend pool3d fwd avg, max operations by f8_t, int8_t types (#1483)

2024-09-17 15:57:10 +02:00

profile_reduce.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profile_softmax.cpp

Padded Generic Kernel Instance (#730)

2023-06-16 23:43:11 -05:00

profile_transpose.cpp

Refactor elementwise kernels (#1222)

2024-04-19 13:31:17 +02:00

profiler_operation_registry.hpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00

profiler.cpp

update copyright headers (#726)

2023-05-31 18:46:57 -05:00