ROCm/composable_kernel
1
0
Fork 0
mirror of https://github.com/ROCm/composable_kernel.git synced 2026-07-03 21:58:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,972 Commits 777 Branches 40 Tags
4c85e10281cbd7d3a7138fdb91123e2fa032bbc6
Commit Graph

312 Commits

Author SHA1 Message Date
joye
4c85e10281 update kernel 2025-06-11 12:38:22 +08:00
joye
7bc604f06a update kernel 2025-06-11 11:10:16 +08:00
joye
b90beae6f0 update kernel 2025-06-11 11:05:49 +08:00
joye
f0a0e79d67 fix compiling errors 2025-06-11 11:01:25 +08:00
joye
0eefe9ac1f update kernel 2025-06-11 10:55:28 +08:00
joye
376e0992ef fix compiling errors; now can pass compilation 2025-06-10 08:34:27 +08:00
joyeamd
4e469f5572 update kernel 2025-06-10 08:11:30 +08:00
joyeamd
d70acd683b update kernel 2025-06-09 21:13:02 +08:00
joyeamd
4c4dd342ed fix compiling errors 2025-06-09 20:44:03 +08:00
joyeamd
9872d2e159 fix some compiling errors 2025-06-09 20:36:55 +08:00
joyeamd
b64a2cfda8 fix some compiling errors 2025-06-09 20:29:30 +08:00
joyeamd
e43bc46629 update kernel 2025-06-09 20:24:47 +08:00
joyeamd
b1d03f7e8a update kernel 2025-06-09 19:37:44 +08:00
joye
0677989d23 Merge branch 'bwd_data_1c_dev' of github.com:ROCm/composable_kernel into bwd_data_1c_dev 2025-06-09 15:34:21 +08:00
joye
51dad1aaca update shader 2025-06-09 15:34:04 +08:00
joyeamd
9b34909b76 add default nullptr 2025-06-09 15:13:40 +08:00
joyeamd
c631001f4e update kernel 2025-06-09 14:51:31 +08:00
joye
48a0cee750 update kernel 2025-06-09 14:41:32 +08:00
joye
1e444a8b27 update kernel 2025-06-09 14:09:33 +08:00
joye
d19a6c0ef0 update kernel 2025-06-09 13:45:31 +08:00
joye
cdf21a7178 update kernel 2025-06-08 20:43:19 +08:00
joye
4fe245e8d5 update kernel; not correctly 2025-06-06 18:35:13 +08:00
joye
a3934c5141 update shader 2025-06-05 16:26:00 +08:00
joye
65df6b65ed update variable to not hard coding 2025-06-05 15:09:35 +08:00
joye
69b6a8b20c update shader 2025-06-04 15:18:00 +08:00
joye
075527783c update shader 2025-06-04 12:21:38 +08:00
joye
37555a8f66 the current best kernel 2025-06-03 15:56:33 +08:00
joye
850b9adbf9 current perf best kernel 2025-06-03 12:56:38 +08:00
joye
d50a7ac6cb update problem shader 2025-06-03 11:08:26 +08:00
joye
f8bc223a58 update kernel to pass 2025-06-03 09:27:57 +08:00
joye
8553458b9b fix some compiling errors 2025-06-03 09:00:47 +08:00
joye
945e3a44ad add another kernel 2025-06-03 08:37:20 +08:00
Bartłomiej Kocot
037764bbc6 Fix grid size calc for bwd wei (#2226) 2025-05-26 16:51:09 +02:00
arai713
5b3430b868 Narrowing error fix for codegen compilation (#2194) 
* removed comment with special characters

* fix for arg/template change after merge from develop

---------

Co-authored-by: Thomas Ning <Thomas.Ning@amd.com>
 2025-05-16 11:11:54 -07:00
Mateusz Ozga
fa3c6811d8 Disable conv for Filter1x1Stride1Pad0 when K or C is even (#2186) 2025-05-16 10:18:47 +02:00
Bartłomiej Kocot
c53b7bd22e Switch to v2 pipeline for grouped conv bwd data (#2181) 
* Change to old pipeline for grouped conv bwd data

* fix

* fix

* fix

* fix

* fix

* fix

* Fix
 2025-05-13 10:14:30 +02:00
Bartłomiej Kocot
6fddb5708c Add grouped conv fwd bias relu instances (#2179) 
* Add grouped conv fwd bias relu instances

* fixes

* fix
 2025-05-09 22:52:34 +02:00
jefyang1
6b1a339b6f Fix grouped conv bwd data tests on gfx950 (#2173) 2025-05-09 09:01:06 -07:00
Andriy Roshchenko
cb27e7c77f Ensure MX GEMM Instances can be Cross-Compiled for Multiple Architectures (#2171) 
* Re-enable MX GEMM instances

* Fix compilation error when building MX GEMM for multiple architectures
 2025-05-08 13:26:03 -06:00
Bartłomiej Kocot
4094ad158a Integrate universal gemm with conv bwd data and add SplitK (#1315) 
* Integrate universal gemm with conv bwd data

* Fix multi d kernel

* Add splitK support

* instances refactor

* instances refactor

* refactor

* fixeS

* fixes

* 16x16 instnaces

* Fixes

* Fix

* Fix

* Fix

* Fix

* Fix

* Fixes

* fix

* fix
 2025-04-28 23:54:49 +02:00
Anton Gorenko
edd92fc546 DeviceGemm_Wmma_CShuffleV3 with BlockGemmPipelineVersion::v3 (#2096) 
* Prepare files for DeviceGemm_Wmma_CShuffleV3

* Implement main part of CShuffleV3 with block pipeline v3 for WMMA

* Remove unused functions and template params for A/B descriptors

* Support both gfx11 and gfx12

* Enable SplitK for gfx12 and disable for gfx11

* Added RowColRow layout for DeviceGemmV2 fp16

* Added more instances for Row, Col, Row data layout

* Added instances for DeviceGemm_Wmma_CShuffleV3, Col, Row, Row data layout

* Added instances for DeviceGemm_Wmma_CShuffleV3, Col, Col, Row data layout

* Added more instances for DeviceGemm_Wmma_CShuffleV3, Row, Row, Row data layout

* Fix formatting

* Add documentation

Based on e5ad48a784

* Enable gemm_universal profiling for gfx11/12

* Add WMMA intrinsics for F8/BF8

* Support F8/BF8 DeviceGemm_Wmma_CShuffleV3, add basic instances

* Add BF16 instances and tests

* Fix test_gemm_universal_wmma_fp8 by adding CK_USE_WMMA_FP8

---------

Co-authored-by: Anca Hamuraru <anca@streamhpc.com>
 2025-04-28 10:14:21 +05:00
lalala-sh
39ba03f25d Moe gemm activation (#2026) 
* fix useless code and remove usless oob

* clang format

* fix coredump in e2e test

* fix2

* fix clang format

* fix output oob

* impl int64 but result not correct

* int64 index ok now

* input output all ok

* fix uint32

* revert v1 test

* use uint32

* mork to support 13w tokens

* moe sorting fix moebuf

* fix merge

* update moe api fix aiter build

* fix buid

* fuse silu

* silu ok

* acale ok

* add silu

* change code

* gemm2 ok

* gufusion compatible ok, fix warnings

* gu fusion for m32 m64 ok

* support bf16 cshuffle

* i4 gemm2 ok

* i4 gemm2 ok and i4 gemm1 build

* 16x16 run ok

* change flops; change cshuffle dtype

* fuse gelu silu act in moe gemm1

* fp8 with act ready

* int4 act ready

* remove useless changes

* remove useless code change

* fix clang format

* add the arch limit of int4 moe gemm

* fuse moe activation

* fix fp8 16x16

* fix no quant case

* fix bugs

* fix fp8 gufusion bug

* remove useless comments

* refine activation code & complete moe example

* fix int8 bugs

* merge tkw1

---------

Co-authored-by: coderfeli <coderfeli@163.com>
Co-authored-by: feli <felix.li@amd.com>
Co-authored-by: illsilin <Illia.Silin@amd.com>
Co-authored-by: root <root@hjbog-srdc-51.amd.com>
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
 2025-04-23 10:35:34 +08:00
lalala-sh
bcf5bb41be enable do top k weights in moe stage1 gemm (#2094) 
* add switch for mul topk weights

* fix bf16/f16 bugs

* complete
 2025-04-18 10:45:49 +08:00
Andriy Roshchenko
213b203a3c MX GEMM - Parameterized Test Template (#2088) 
* Tests for MX FP8 GEMM

* Improve documentation
 2025-04-16 19:56:00 -06:00
Andriy Roshchenko
da54464cce MX GEMM - Add MX BF8 example (#2071) 
* Add MX GEMM example for MX BF8

* Verified MX FP8 with 16x16x128 scale builtin

* Verify MX BF8 GEMM with BF16 output
 2025-04-16 15:25:02 -06:00
Andriy Roshchenko
7106976a72 MX GEMM - New GEMM pipeline for MX data types (#2059) 
* Allow selection of mfma_scale instructions

* Read B tensor from LDS to VGPR in chunks of 16 in MFMA order

* Add constexpr and synchronize return type for `get_exponent_value`

* Pass scales by reference and add comments to `mfma_scale_f32_32x32x64`

* Add support for microscaling instructions in `XdlopsGemm`

* Fix `mfma_scale_f32_16x16x128f8f6f4` wrapper

* Remove software implementation of MX GEMM

* Make interface of `intrin_mfma_scale_f32_16x16x128f8f6f4<16, 16>` consistent with the other scale instruction

* Update README

* Updated CHANGELOG

* Remove unused static methods
 2025-04-15 17:17:07 -06:00
Mingtao Gu
56378f810f CK pk_i4_t test failures fix (SWDEV-518629) (#2075) 
* fix pk_i4_v3 tests failures in Unbuntu env.

* fix pk_i4_t tests failure on Unbuntu issues.

* some fixed.

---------

Co-authored-by: mtgu0705 <mtgu@amd.com>
 2025-04-14 16:58:57 +08:00
Illia Silin
0d4f145078 Fix build issues for multiple targets. (#2077) 
* build for multiple targets on gfx942

* add missing ignore statements
 2025-04-11 12:12:53 -07:00
Illia Silin
572cd820ce Split env.hpp header from the ck.hpp header. (#2049) 
* split env.hpp out of main headers

* fix namespace logic
 2025-04-03 15:30:21 -07:00
aledudek
9329432f6c Post-merge changes for fully async args copy in ck grouped gemm (#1991) 
* Post-merge changes for fully async args copy in ck grouped gemm

* Post-merge documentation and naming changes

* Build fix and updated changelog

* Revised comments
 2025-04-03 13:35:43 +02:00