Shaojie WANG 45adb736e7 Padding for attention: bmm+scale+softmax+bmm kernel (#385) ...

* add padding algo for bmm+scale+softmax+bmm. Version for verification

* remove verification code

* remove comments

* add padded bmm scale softmax bmm example

* format

* refactor

* add comments for usages of padding bmm+scale+softmax+bmm

Co-authored-by: Chao Liu <lc.roy86@gmail.com>

2022-08-30 11:01:37 -05:00

..

block_to_ctile_map.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_2d_multiple_reduction_multiblock.hpp

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

gridwise_2d_multiple_reduction_threadwise.hpp

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

gridwise_2d_reduction_multiblock.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_2d_reduction_threadwise.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_batched_gemm_gemm_xdl_cshuffle_v1.hpp

Attention with output permutation (#370)

2022-08-23 14:52:56 -05:00

gridwise_batched_gemm_softmax_gemm_xdl_cshuffle_v1.hpp

Padding for attention: bmm+scale+softmax+bmm kernel (#385)

2022-08-30 11:01:37 -05:00

gridwise_contraction_dlops_v1r2.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_elementwise_1d.hpp

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

gridwise_gemm_bias_add_reduce_xdl_cshuffle_v1.hpp

external api for gemm + layernorm (#285)

2022-06-27 14:25:10 -05:00

gridwise_gemm_dl_v1r3.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_dlops_v1r2.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_dlops_v2.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_dlops_v3.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_multiple_d_multiple_r_xdl_cshuffle.hpp

Refactor the design of DeviceGemmMultipleDMultipleR_Xdl_CShuffle (#378)

2022-08-24 10:12:54 -05:00

gridwise_gemm_multiple_d_xdl_cshuffle.hpp

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

gridwise_gemm_pipeline_v1.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_pipeline_v2.hpp

GEMM pipeline v2 (#317)

2022-07-08 15:55:14 -05:00

gridwise_gemm_reduce_xdl_cshuffle_v1.hpp

external api for gemm + layernorm (#285)

2022-06-27 14:25:10 -05:00

gridwise_gemm_xdl_cshuffle_v1.hpp

GEMM pipeline v2 (#317)

2022-07-08 15:55:14 -05:00

gridwise_gemm_xdl_layernorm_cshuffle_v1.hpp

Single-kernel GEMM + layernorm (#263)

2022-07-01 01:38:00 -05:00

gridwise_gemm_xdlops_bwd_weight.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_xdlops_skip_b_lds_v1.hpp

Fused GEMM+GEMM (#351)

2022-08-13 09:18:58 -05:00

gridwise_gemm_xdlops_v2r3.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_xdlops_v2r4.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_xdlops_v2r4r2.hpp

Add examples of batched/grouped/SplitK Gemm for int8/bfp16/fp16/fp32 (#361)

2022-08-23 14:41:56 -05:00

gridwise_gemm_xdlops_v3r1.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_xdlops_v3r2.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_gemm_xdlops_v3r3.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_layernorm_naive_variance.hpp

Layernorm welford (#346)

2022-08-13 09:43:18 -05:00

gridwise_layernorm_welford_variance.hpp

Layernorm welford (#346)

2022-08-13 09:43:18 -05:00

gridwise_set_buffer_value.hpp

add license in file (#303)

2022-06-24 23:32:43 -05:00

gridwise_set_multiple_buffer_value.hpp

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

gridwise_softmax.hpp

fix standalone softmax race condition around blockwise reduction (#323)

2022-07-14 22:52:45 -05:00