ROCm/composable_kernel
1
0
Fork 0
mirror of https://github.com/ROCm/composable_kernel.git synced 2026-07-07 23:57:14 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,202 Commits 776 Branches 40 Tags
70d57ca8b96bfdfd8d2da0648750d8465fe484ff
Commit Graph

566 Commits

Author SHA1 Message Date
Ville Pietilä
70d57ca8b9 Remove separate packed cast step. 2025-09-03 10:57:16 +00:00
Ville Pietilä
d56d7bc821 Use fused packed cats. 2025-09-02 12:00:21 +00:00
Ville Pietilä
d22ec6633a Fused packed cast improvements. 2025-09-01 14:57:42 +00:00
Ville Pietilä
c69539fe3c Optimize LDS write order for packed cast. 2025-08-28 15:01:55 +00:00
Ville Pietilä
9f66d9fbca Packed cast improvement. 2025-08-27 14:32:00 +00:00
Ville Pietilä
b82e68c45f Bug fix. 2025-08-27 14:30:13 +00:00
Ville Pietilä
f1a7cfba26 Packed cast imrovement. 2025-08-27 14:23:05 +00:00
Ville Pietilä
90b90dff08 Vectorized packed cast. 2025-08-27 13:29:38 +00:00
Ville Pietilä
54302c6f77 Remove obsolete version of the packed cast. 2025-08-27 11:28:55 +00:00
Ville Pietilä
481df169f2 Add packed cast to gridwise gemm multi d. 2025-08-27 11:27:03 +00:00
Ville Pietilä
6092643e9b Performat packed cast implementation. 2025-08-27 10:10:18 +00:00
Ville Pietilä
cfbd669455 WIP: Vectorized access. 2025-08-26 12:43:38 +00:00
Ville Pietilä
2302ea9bc6 Add the vectorized option for packed cast. 2025-08-26 12:31:01 +00:00
Ville Pietilä
905cfb6623 Use thread scratch buffer in bf16 conversion. 2025-08-26 09:33:50 +00:00
Ville Pietilä
1b858936cb WIP: Packed cast using threadwise sratch. 2025-08-25 15:31:50 +00:00
Ville Pietilä
4e7f9f7908 Fix packed cast tensor slice transfer. 2025-08-22 10:26:59 +00:00
Ville Pietilä
de93a48b04 Add back the separate packed cast step. 2025-08-20 11:31:07 +00:00
Ville Pietilä
6fbe1895f1 Code clean-up. 2025-08-20 08:19:13 +00:00
Ville Pietilä
0d34572f20 Code clean-up. 2025-08-19 18:37:01 +00:00
Ville Pietilä
c0b8f66674 Add packed cast pipeline into gridwise gemm xdlops bwd weight. 2025-08-18 06:10:14 +00:00
Ville Pietilä
00a3ce734a Integrate new packed cast threadwise tensor slice transfer into gridwise gemm pipelines. 2025-08-15 12:06:44 +00:00
Ville Pietilä
51af3d7bac Fix a bug in the packed cast threadwise transfer. 2025-08-15 10:41:06 +00:00
Ville Pietilä
62c66a7d9c WIP: packed bf16 cast v3. 2025-08-14 12:39:18 +00:00
Ville Pietilä
938ff298b4 Add more unit tests. 2025-08-14 11:33:02 +00:00
Ville Pietilä
ade741dd45 WIP: PackedCast v3. 2025-08-13 15:13:35 +00:00
Ville Pietilä
50e318e072 Fix logging. 2025-08-12 15:53:00 +00:00
Ville Pietilä
ae4c727bc5 Add packed bf16 cast for universal GEMM. 2025-08-12 15:52:49 +00:00
Ville Pietilä
cee7644c85 Working version 2 of the packed cast. 2025-08-12 12:46:01 +00:00
Ville Pietilä
6148d1c75f WIP: Packed cast v2. 2025-08-11 15:18:30 +00:00
Ville Pietilä
c675563468 Addlogging and specific unit tests for bf16 and gfx950. 2025-08-08 08:59:12 +00:00
Ville Pietilä
4b8a559da9 Fixed packed_cast implementation for slice access. 2025-08-06 11:04:29 +00:00
Ville Pietilä
44202b9d32 WIP: Integration of packed cast into gridwise_gemm_xdl_cshuffle_conv_v3. 2025-08-05 15:12:36 +00:00
Ville Pietilä
e92c0bf68e Initial integaration of packed cast. 2025-08-04 15:34:35 +00:00
Ville Pietilä
9769fa68a7 Packed BF16 cast with asm volatile. 2025-08-01 13:22:41 +00:00
Ville Pietilä
e962a41638 Automatic deduction of split-K value for grouped convolution (#2491) 
* Split-K autodeduction for DeviceGroupedConvBwdWeight_Xdl_CShuffle and DeviceGroupedConvBwdWeight_Xdl_CShuffleV3.

* Split-K autodeduction for DeviceGroupedConvBwdWeightTwoStage_Xdl_CShuffle.

* Use simple best occupancy model to calculate the split-K.

* Handle split-K autodeduction in explicit gemm conv.

* Add unit tests for split-K autodeduction.

* Remove oversubscription.

* Small fixes.

* Added split-K autodeduction for DeviceGroupedConvBwdWeightMultipleD_Xdl_CShuffle.

* Run clang formatting.

* Fix error handling in the conv profiler.

* Add missing documentation for the autodeducted split-K values.

* Add split-K autodeduction to DeviceGroupedConvBwdWeight_Explicit_Xdl solver.

* Fix clang formatting and split-K profiler documentation.

* Rename max_occupancy value variable.

* Calculate grid size for split-K autodeduction directly from input array shapes and template params.

---------

Co-authored-by: Ville Pietilä <>
 2025-07-31 12:08:45 +02:00
Bartłomiej Kocot
5b244105d9 Enable multiple D for grouped conv fwd large tensors (#2572) 2025-07-28 22:39:07 +02:00
linqunAMD
0782ee8eb3 Remove !defined(__HIP_DEVICE_COMPILE__) in CK kernel (#2564) 
* Remove HIP_COMPILE_DEVICE

* add missing files

* fix clang format

---------

Co-authored-by: Lin, Qun <Quentin.Lin+amdeng@amd.com>
 2025-07-28 13:01:07 -07:00
Illia Silin
504b101da3 upgrade from clang-format-12 to clang-format-18 (#2568) 
* upgrade to clang-format-18

* update to clang-format-18 in pre-commit-config
 2025-07-28 11:34:07 -07:00
Adam Osewski
c8eb2f995c Add v3 support for Groupd fwd conv+bias+clamp & ckProfiler (#2463) 
* Add logging to IsSupported.

* Less casting in AddClamp

* Conv+bias+clamp instances & profiler BF16

* Fix 3D instances & run just 1x for verification.

* :Run just once for verification conv fwd.

* ckProfiler conv fwd clampwq

* Remove exec bit & formatting

* Add support for MultiD for grouped conv fwd v3.

* Enable 2Lds.

* clean

* align instances

* align instances

* profiler fixes

* Fixes

* fix

* fix

---------

Co-authored-by: Adam Osewski <root@quanta-ccs-aus-f01-19.cs-aus.dcgpu>
Co-authored-by: Bartłomiej Kocot <barkocot@amd.com>
 2025-07-25 10:34:31 +02:00
Enrico Degregori
b01a27ff22 Support b_scale: (#2350) 
- extend pipeline v1 and v3
 - add instances
 - add tests
 - add example

Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
 2025-07-24 18:49:58 -07:00
Mingtao Gu
0198257d79 [CK] Fixed MPerBlock=32 build issue for MXFP4 GEMM decode (#2512) 
* added MPerBlock=32 for MXFP4 GEMM decode

* added two instance for M>128 scenario.

* added 1 instance

* format

---------

Co-authored-by: mtgu0705 <mtgu@amd.com>
Co-authored-by: felix <felix.li@amd.com>
 2025-07-18 14:35:54 +08:00
linqunAMD
fbd9f32abe [CK][CONV] Support NCHW in class DeviceGroupedConvBwdDataMultipleD_Xdl_CShuffle_v1 (#2459) 
1. Port NCHW support from ConvFwd (#2375) to conv bwd data
2. Add new instance device_grouped_conv_bwd_data_xdl_f16_nchw_instances for nchw

Co-authored-by: azhuang <anzhong.huang@amd.com>
 2025-07-17 08:19:57 +08:00
Illia Silin
a4bf78ac0e replace obsolete warpSize system variable with the new one (#2496) 2025-07-16 07:39:15 -07:00
huaiguxu
c1badfd30c Handle moe_fp8 no-mainloop cases. Supprese no-mainloop check (#2438) 
Co-authored-by: felix <felix.li@amd.com>
 2025-07-16 15:44:34 +08:00
Andriy Roshchenko
054f85ab7c MX GEMM - FP6 Example (#2419) 
Adds support for MX FP6 data type in MX GEMM block pipeline version v1.
Provides an example of MX FP6 GEMM algorithm.

---------

Co-authored-by: OscarXu <huaiguxu@amd.com>
Co-authored-by: aska-0096 <haocwang@amd.com>
Co-authored-by: mtgu0705 <mtgu@amd.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: lalala-sh <Jiaxing.Wen@amd.com>
Co-authored-by: valarLip <340077269@qq.com>
Co-authored-by: Ding, Yi <yi.ding@amd.com>
Co-authored-by: feifei14119 <feiw@amd.com>
Co-authored-by: Lin, Qun <qlin@amd.com>
Co-authored-by: joye <joye@amd.com>
 2025-07-07 10:33:26 -06:00
Mingtao Gu
7998ae8969 [CK] Mxfp4 moe blockscale buf2lds version support (#2455) 
* change cshuffle size

* added mxfp4 moe async buffer loading without B preshuffle

* added mx moe B shuffling + scale shuffling (async loads)

* minor fix

---------

Co-authored-by: mtgu0705 <mtgu@amd.com>
 2025-07-06 15:42:00 +08:00
Adam Osewski
3d70c638d1 Always force output clearing for grouped conv bwd data (#2446) 
* Always force output clearing

* dont run set zero for residual

---------

Co-authored-by: Bartlomiej Kocot <barkocot@amd.com>
 2025-07-04 07:49:52 -06:00
Vidyasagar Ananthan
2e971eff90 Removing reference to undefined parameter for ignore statement. (#2447) 2025-07-03 20:10:29 -07:00
damien-lejeune
1183824573 Fix clang in ck develop branch (#2445) 
Co-authored-by: Damien Lejeune <damien.lejeune@amd.com>
 2025-07-02 10:07:47 -06:00
chenjun
74a34e0f50 fix KPerBlock = 64 a8w8 bpreshulle gemm build fail in gfx950 (#2437) 
Co-authored-by: valarLip <340077269@qq.com>
 2025-07-02 19:12:07 +08:00