Anthony Chang 9287b7c6b3 Grouped batched attention + permute (#412) ...

* grouped attn without batch validates; now move toward grouped batched attn

* grouped batched attention

* working

* remove debug logging

clean up

clean up

* reintroduce g_ prefix back to host tensor variables

* format

* rename file

* restore old file

* rename

* consolidate padded/non-padded attention example

* harmonize padding specialization in attn examples

2022-09-19 16:09:44 -05:00

..

01_gemm

add more datatype to gemm+gemm and conv+conv example (#397)

2022-09-01 09:31:17 -05:00

02_gemm_bilinear

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

03_gemm_bias_relu

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

04_gemm_add_add_fastgelu

Add example of Gemm + AddAddFastGelu (data type: int4) (#369)

2022-08-23 10:38:41 -05:00

09_convnd_fwd

add more datatype to gemm+gemm and conv+conv example (#397)

2022-09-01 09:31:17 -05:00

10_convnd_fwd_multiple_d_multiple_reduce

Add examples of Conv + reduction (data type: int4, int8, bf16, fp16, fp32) (#380)

2022-08-31 16:32:17 -05:00

12_reduce

Add int4 reduction examples (#372)

2022-08-25 16:58:48 -05:00

13_pool2d_fwd

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

14_gemm_xdl_requant_relu_requant

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

15_grouped_gemm

GEMM batched/splitK/cgemm/grouped int4 examples (#383)

2022-08-25 17:19:15 -05:00

16_gemm_multi_d_multi_reduces

Gemm reduce examples int4/int8/fp32/bf16 (#368)

2022-08-30 11:38:26 -05:00

17_convnd_bwd_data

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

18_batched_gemm_reduce

Fused attention (#345)

2022-08-13 00:16:14 -05:00

19_binary_elementwise

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

20_convnd_bwd_weight

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

21_gemm_layernorm

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

22_cgemm

GEMM batched/splitK/cgemm/grouped int4 examples (#383)

2022-08-25 17:19:15 -05:00

23_softmax

Softmax client example (#396)

2022-09-06 12:22:48 -05:00

24_batched_gemm

GEMM batched/splitK/cgemm/grouped int4 examples (#383)

2022-08-25 17:19:15 -05:00

25_gemm_bias_e_permute

add g; fixed strides (#355)

2022-08-12 15:22:39 -05:00

26_contraction

Clean up conv example, Instances, profiler and test (#324)

2022-07-29 18:19:25 -05:00

27_layernorm

Layernorm welford (#346)

2022-08-13 09:43:18 -05:00

28_grouped_gemm_bias_e_permute

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

29_batched_gemm_bias_e_permute

Add batched/grouped_gemm contraction deviceOps (#349)

2022-08-10 12:20:29 -05:00

30_grouped_convnd_fwd_bias_relu_add

Conv bwd data multiple d (#404)

2022-09-19 11:25:28 -05:00

31_batched_gemm_gemm

add more datatype to gemm+gemm and conv+conv example (#397)

2022-09-01 09:31:17 -05:00

32_batched_gemm_scale_softmax_gemm

Grouped batched attention + permute (#412)

2022-09-19 16:09:44 -05:00

33_multiple_reduce

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

34_batchnorm

Batchnorm-forward and Batchnorm-infer Implemented using generic kernels (#320)

2022-08-15 10:11:02 -05:00

35_splitK_gemm

GEMM batched/splitK/cgemm/grouped int4 examples (#383)

2022-08-25 17:19:15 -05:00

36_sparse_embedding

embedding fuse layernorm (#405)

2022-09-09 10:41:15 -05:00

37_batched_gemm_add_add_relu_gemm_add

batched_gemm + multiple_d + gemm + multiple_d (#394)

2022-09-14 17:54:18 -05:00

38_grouped_conv_bwd_data_bias_relu

Conv bwd data multiple d (#404)

2022-09-19 11:25:28 -05:00

41_grouped_conv_conv_fwd

add more datatype to gemm+gemm and conv+conv example (#397)

2022-09-01 09:31:17 -05:00

CMakeLists.txt

Conv bwd data multiple d (#404)

2022-09-19 11:25:28 -05:00