ROCm/composable_kernel
1
0
Fork 0
mirror of https://github.com/ROCm/composable_kernel.git synced 2026-05-14 18:17:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
473 Commits 743 Branches 38 Tags
ea6fa92eea7aee6addffdf9d2f44e56b6ea663ef
Commit Graph

39 Commits

Author SHA1 Message Date
Jing Zhang
ea6fa92eea updated bfloat16_to_float 
[ROCm/composable_kernel commit: 89e1ebd4d5]
 2021-11-16 18:01:25 +00:00
zjing14
456f5306df Add bfp16/int8 support into XDL GEMM operator (#50) 
* init StaticBufferV2

* clean

* adopt old output stage for staticBufferV2

* clean

* remove hack

* clean

* clean

* add parameters

* clean code

* move c_buffer alloc into blockwise gemm

* add adaptors for m/n_thread_data_on_grid

* tweak gemm

* adjust blockwise_gemm_xdlops

* tweak

* update conv

* update script

* adding bwd 1x1

* update script

* adding 1x1 bwd

* debugging bwd 1x1 failure

* update script

* update script

* test

* test v100

* add bf16_1k

* clang-format

* clean

* add bfp16 for gfx908

* add verification

* clean up

* clean code

* restore bfl16

* clean

* add bfp16 support into gemm_driver

* apply new generator to other drivers

* add int8 support

* cleanb

* clean

* clean

* clean

Co-authored-by: Chao Liu <chao.liu2@amd.com>
Co-authored-by: Chao Liu <lc.roy86@gmail.com>
Co-authored-by: root <root@hayabusa6111.amd.com>

[ROCm/composable_kernel commit: 3737bb039a]
 2021-11-15 10:24:39 -06:00
Chao Liu
8791d26e52 FP16 data in-register transpose (#41) 
* start fixing 16bit data packing

* adding StaticTensor

* adding StaticTensor

* adding StaticTensor

* add missing constexpr

* adding static tensor

* adding static tensor

* adding transpose

* add inline asm for transpose 2x2 of half_t

* add general transpose_vectors(), but have unnecessary register initialization using v_mov

* fix unnecessary register initialization in transpose_vector by using more pass-by-reference

* add hardcoded logic for NHWC wrw

* improve asm for v_pack

* make ThreadwiseTensorSliceTransfer_v3r2 support any tensor

* tweak

* reorganize file

[ROCm/composable_kernel commit: b491ebf384]
 2021-11-15 10:05:58 -06:00
Chao Liu
2f5ccb68f5 ckProfiler and device-level XDL GEMM operator (#48) 
* add DeviceGemmXdl

* update script

* fix naming issue

* fix comment

* output HostTensorDescriptor

* rename

* padded GEMM for fwd v4r4r4 nhwc

* refactor

* refactor

* refactor

* adding ckProfiler

* adding ckProfiler

* refactor

* fix tuning parameter bug

* add more gemm instances

* add more fp16 GEMM instances

* fix profiler driver

* fix bug in tuning parameter

* add fp32 gemm instances

* small fix

* refactor

* rename

* refactor gemm profiler; adding DeviceConv and conv profiler

* refactor

* fix

* add conv profiler

* refactor

* adding more GEMM and Conv instance

* Create README.md

Add build instruction for ckProfiler

* Create README.md

Add Readme for gemm_xdl example

* Update README.md

Remove build instruction from top most folder

* Update README.md

* clean up

[ROCm/composable_kernel commit: e823d518cb]
 2021-11-14 11:28:32 -06:00
ltqin
1f6ca26819 [Bug Fix] GridwiseGemm_bk0mk1_bk0nk1_mn_xdlops_v2r4 loop issue (#44) 
* change method computering kpad

* remove unusing variable: batchlen

* change KPerBlock to K0PerBlock

* fix bug for k0 == k0perblock

* fix bug for get k0 index

* use math::integer_divide_ceil

Co-authored-by: ltqin <letaoqin@amd.com>
Co-authored-by: Chao Liu <chao.liu2@amd.com>

[ROCm/composable_kernel commit: 6014185ac6]
 2021-10-27 09:39:18 -05:00
Chao Liu
3fcbcb776c fix bug in gridwise gemm xdlops v2r3 (#45) 
[ROCm/composable_kernel commit: d5297abae9]
 2021-10-21 16:42:24 -05:00
ltqin
0d74bff825 add nchw atomic , nhwc and nhwc atomic method for backward weight (#30) 
* add add new algorithm from v4r4r2

* program once issue

* add split k functiion

* redefine code

* add a matrix unmerge

* add b matrix unmerge k0

* trans a and b to gridegemm

* nhwc init

* no hacks and vector load

* add hacks

* modify some parameter

* fix tuning prometer for fp32

* fix tuning prometer for fp16

* start change gridwise k split

* init ok

* revome a b matrix k0mk1 desc in grid

* carewrite lculate gridsize

* add kbatch to CalculateBottomIndex

* remove some unused funtion

* add clear data function before call kernel

* out hacks

* in hacks

* rename device convolution file and function name

* modify kBatch value

* fix some tuning code

* start from v4r4 nhwc

* nhwc atomic is able to run

* just for fp32

* enable nchw atomic

* tweak

* tweak

* re-arrange gridwise gemm hot loop for wrw

* add wrw v4r5

* v4r4r5 fp16

* v4r4r4 fp16

* v4r4r2 fp16

* V4R4R4XDLNHWC fp16

* V4R4R2XDLATOMICNCHW fp16

* adjust for fp16

* input gridsize

* change kbatch to gridsize

* testing wrw

* clean up

* k_batch to gridsize

* fix bug

* wrw v4r4r4 kbatch change to gride size

* wrw v4r4r2 kbatch change to gride size

* after merge , change gridwise gemm v2r4

* change MakeCBlockClusterAdaptor

* other method use new gridwise gemm

* clean up

* chapad method nge to make_right_pad_transform

* kbatch out from transform function

* clean up and fix bug

* fix bug

* using function type reduce template parameters

* using auto replace define fuction type

* clean up

Co-authored-by: ltqin <letaoqin@amd.com>
Co-authored-by: Chao Liu <chao.liu2@amd.com>
Co-authored-by: Jing Zhang <jizhan@amd.com>

[ROCm/composable_kernel commit: fd49ff8080]
 2021-10-19 18:42:34 -05:00
Chao Liu
720cf3d6b2 Tweak GEMM kernel (#38) 
* add parameters

* tweak gemm

* tweak

* update conv

* update script

* adding bwd 1x1

* update script

* adding 1x1 bwd

* debugging bwd 1x1 failure

* update script

* update script

* test

* test v100

* clean up

[ROCm/composable_kernel commit: b3e8d57d51]
 2021-10-06 11:12:36 -05:00
Chao Liu
079adb1e7d GEMM driver and kernel (#29) 
* add gemm driver

* tweak

* add gemm kernel: mk_kn_mn and km_kn_mn

* tweak

* add GEMM km_nk_mn

* fix comment

[ROCm/composable_kernel commit: 19613902b5]
 2021-09-05 12:41:28 -05:00
ltqin
2f4f6427f5 Backward weight v4r4r2 with xdlops (#18) 
* start

* modify transformat

* modify device convolutiion

* modify host

* added host conv bwd and wrw

* remove bwd, seperate wrw

* clean

* hacall k to zero

* out log

* fixed

* fixed

* change to (out in wei)

* input hack

* hack to out

* format

* fix by comments

* change wei hacks(wei transform has not merge)

* fix program once issue

* fix review comment

* fix vector load issue

* tweak

Co-authored-by: ltqin <letaoqin@amd.com>
Co-authored-by: Jing Zhang <jizhan@amd.com>
Co-authored-by: Chao Liu <chao.liu2@amd.com>

[ROCm/composable_kernel commit: 627d8ef35a]
 2021-08-30 22:49:17 -05:00
zjing14
14e4d7ba7d Xdlops refactor fix (#22) 
* added constexpr ahead of adptor; clean unused driver; rename M/NPerWave to M/NPerXDL

* fixed bwd

* fixed comment

[ROCm/composable_kernel commit: 9d3f634a3c]
 2021-08-23 11:22:10 -05:00
zjing14
6dde0995c5 refactor dynamic xdlops iGemm (#13) 
* xdlops refactor

* fixed commnt

* clean xdlops_gemm

* add make c into xldops-gemm

* change mfma_info

* refactor xdlops, hide c desc

* clean

* clean

* clean

* apply hacks changes to v4r4r4_nhwc

* rename hacks and use single stage adapter

* enable fp16 mfma

[ROCm/composable_kernel commit: a2ad6d3531]
 2021-08-19 09:54:10 -05:00
zjing14
f8e4daa52c Added host_conv_wrw for verification (#15) 
* added host conv wrw

[ROCm/composable_kernel commit: ba6f79a75e]
 2021-08-19 01:00:41 -05:00
Chao Liu
65e8349051 DynamicBuffer, StaticBuffer, amd_buffer_load support customized value for invalid element 
[ROCm/composable_kernel commit: a91b68dfcb]
 2021-08-13 23:40:19 +00:00
Chao Liu
04ed8ddf43 compiler parameter use stream 
[ROCm/composable_kernel commit: 0834bc7635]
 2021-08-13 01:05:14 +00:00
Chao Liu
9f40048d1c make innner product compatiable on gfx900 
[ROCm/composable_kernel commit: f2ac7832c6]
 2021-08-11 09:42:53 -05:00
Chao Liu
f7df8c7eea rename 
[ROCm/composable_kernel commit: 4e57b30a6a]
 2021-08-11 00:08:42 +00:00
Chao Liu
1e312fef12 rename 
[ROCm/composable_kernel commit: c03045ce2d]
 2021-08-10 23:45:36 +00:00
Chao Liu
c825eb6b1c fix kernel filename 
[ROCm/composable_kernel commit: 2c48039d0e]
 2021-08-10 22:15:23 +00:00
Chao Liu
a7c943abab fix clang warning suppression 
[ROCm/composable_kernel commit: ddd49ec9e7]
 2021-08-10 06:20:24 +00:00
Chao Liu
d49e0ddcb2 vector/scalar pointer cast use c-style pointer cast instead of reinterpret_cast 
[ROCm/composable_kernel commit: 4f566c6221]
 2021-08-10 05:55:20 +00:00
Chao Liu
314b9d78ef add c-style pointer cast 
[ROCm/composable_kernel commit: 172036d728]
 2021-08-10 00:01:52 -05:00
Chao Liu
d4b35bd09f tidy 
[ROCm/composable_kernel commit: 76f3131939]
 2021-08-09 18:49:59 -05:00
Chao Liu
cb2edf2100 tidy 
[ROCm/composable_kernel commit: d18428901e]
 2021-08-09 18:20:02 -05:00
Chao Liu
4771cfa340 tidy 
[ROCm/composable_kernel commit: f885c131d8]
 2021-08-09 22:13:47 +00:00
Chao Liu
b14b5d3371 tidy 
[ROCm/composable_kernel commit: c3efeb5e20]
 2021-08-09 19:32:07 +00:00
Chao Liu
9c589af829 tidy 
[ROCm/composable_kernel commit: 56fc0842b3]
 2021-08-09 19:27:49 +00:00
Chao Liu
e8def0e778 tidy 
[ROCm/composable_kernel commit: 54fba515b3]
 2021-08-09 17:33:32 +00:00
Chao Liu
9e2c3c7765 tidy 
[ROCm/composable_kernel commit: e62bae7a4a]
 2021-08-09 15:11:35 +00:00
Chao Liu
cba13cb6b3 fix 
[ROCm/composable_kernel commit: 61487e0a00]
 2021-08-07 02:31:19 +00:00
Chao Liu
5ed1b840a0 remove online compilation from CK 
[ROCm/composable_kernel commit: ae98b52ad8]
 2021-08-07 00:51:05 +00:00
Chao Liu
5856acc10f refactor 
[ROCm/composable_kernel commit: cb95421311]
 2021-08-06 22:17:51 +00:00
Chao Liu
f017e34483 refactor 
[ROCm/composable_kernel commit: 7b1ec41e5b]
 2021-08-06 20:50:01 +00:00
Chao Liu
9eb35eec87 refactor 
[ROCm/composable_kernel commit: 49c33aaea7]
 2021-08-06 19:59:53 +00:00
Chao Liu
041c48a06c rename 
[ROCm/composable_kernel commit: 54b3e73d17]
 2021-08-06 18:07:15 +00:00
Chao Liu
e2352d83a9 update to clang-format-10 
[ROCm/composable_kernel commit: 82fae390fb]
 2021-07-30 16:37:00 -05:00
Chao Liu
f864700e84 fix building 
[ROCm/composable_kernel commit: 6a1bc5939c]
 2021-07-27 13:12:43 -05:00
Chao Liu
cb9222657c [MIOpen Downstream] Initial MIOpen integration (#52) 
* update online kernel wrapper bundle all descriptors in a tuple

* change __CONSTANT__ to CONSTANT

* rename

* adding tuning

* added IsValidCompileParameter

* reorginze

* adding tunable for fp16 and int8

* fix kernel compile warning and bug fixes

* suppress warning about cast CONSTANT (address space 4) pointer

* fix building issue

[ROCm/composable_kernel commit: f63a23acb1]
 2021-07-27 00:02:27 -05:00
Chao Liu
b6c15f3eec reorganize files to prepare for MIOpen integration (#51) 
* change olc cmake

* adding online compile to fwd-v4r5r2

* update scripts

* remane fwd-v4r5r2 to fwd-v6r1

* clean up

[ROCm/composable_kernel commit: 1264925422]
 2021-07-18 00:43:05 -05:00