ROCm/composable_kernel
1
0
Fork 0
mirror of https://github.com/ROCm/composable_kernel.git synced 2026-05-14 10:09:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,863 Commits 742 Branches 38 Tags
807501ac3d5ab19a2e33fef93bff2d136d251d31
Commit Graph

1863 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
carlushuang
807501ac3d [CK_TILE] optimize moe sorting kernel, boost large context case up to 20x (#2153) 
* combine 2-3 as single stage

* support zeroing

* improve long tokens

* update specialization

* b16 ws

* 8bit topk optimize

* update 15 example

[ROCm/composable_kernel commit: 4e9b76f88c]
 2025-05-06 17:32:07 +08:00
Muhammed Emin Ozturk
d22cc72f32 Fix failure in test_batched_gemm_softmax_gemm_permute for lower resource devices (#2117) 
* Problematic test case are analyzed and turned off for lower resource GPUs

* update device info

* Update test_batched_gemm_softmax_gemm_permute_bf16_xdl.cpp

* Update test_batched_gemm_softmax_gemm_permute_fp16_xdl.cpp

* Update test/batched_gemm_softmax_gemm_permute/test_batched_gemm_device_utils.hpp

Co-authored-by: John Afaganis <john.afaganis@amd.com>



[ROCm/composable_kernel commit: b8fa27bfef]
 2025-05-05 13:12:22 -07:00
jakpiase
354a09f55c [CK_TILE] Remove scratch usage from universal gemm (#2001) 
* moves kbatch condition outside of kernel

* add reviewer comments

* fixes

* fix tests

* fixes after review

---------

Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>

[ROCm/composable_kernel commit: 0bcb804ad0]
 2025-05-05 18:46:44 +02:00
Andriy Roshchenko
bde3b6c5ad Restrict MX GEMM instantiation to GFX950 arch (#2157) 
[ROCm/composable_kernel commit: 79beaacdd1]
 2025-05-05 08:18:22 -07:00
dependabot[bot]
362e51de66 Bump rocm-docs-core[api_reference] from 1.18.2 to 1.18.4 in /docs/sphinx (#2161) 
Bumps [rocm-docs-core[api_reference]](https://github.com/ROCm/rocm-docs-core) from 1.18.2 to 1.18.4.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.18.2...v1.18.4)

---
updated-dependencies:
- dependency-name: rocm-docs-core[api_reference]
  dependency-version: 1.18.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/composable_kernel commit: c4e4e592c1]
 2025-05-05 07:29:07 -07:00
Khushbu Agarwal
5d0b1b733c mfma_32x32x64_fp8/bf8 (#2148) 
* support for mfma_32x32x64_fp8

* clang-formatted

* Fixing sparsity in codegen

[ROCm/composable_kernel commit: d58f2b8bd0]
 2025-05-01 13:36:24 -07:00
Illia Silin
9063797aee re-enable ck4inductor tests by default (#2155) 
[ROCm/composable_kernel commit: 619fba3134]
 2025-05-01 12:37:27 -07:00
Andriy Roshchenko
e70035cf5a MX GEMM - Add FP8 GEMM Tests for Different Layouts (#2152) 
* Add gemm_mx_fp8_bf8 example with row-major B

* Add more overloads of MX MFMA instructions

* Add MK_KN (RRR) tests

* Add KM_NK (CCR) tests

* Add more problem sizes to Large tests

* Add test_gemm_mx to the list of regression tests

[ROCm/composable_kernel commit: 79b0bfeb41]
 2025-05-01 11:55:48 -06:00
Illia Silin
e6e687d8a7 add write permissions in workspace (#2154) 
[ROCm/composable_kernel commit: b9d17bdb11]
 2025-05-01 07:04:57 -07:00
Aviral Goel
7cedc33703 Add documentation for ck_tile::array<T,N> (#2078) 
* addded documentation for ck_tile::array<T,N>

* clang format fix

* spelling errros

Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>

* spelling errros

Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: spolifroni-amd <Sandra.Polifroni@amd.com>

* Revert "spelling errros"

This reverts commit 4179e7d193.

* Revert "spelling errros"

This reverts commit 3f90733dbe.

---------

Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>
Co-authored-by: spolifroni-amd <Sandra.Polifroni@amd.com>
Co-authored-by: John Afaganis <john.afaganis@amd.com>

[ROCm/composable_kernel commit: 1d8ef40760]
 2025-04-30 16:43:36 -07:00
spolifroni-amd
9114925d46 updated Doxyfile and added the class list (#2147) 
* updated Doxyfile and added the class list

* Update Doxyfile

[ROCm/composable_kernel commit: cfae863431]
 2025-04-30 14:58:40 -07:00
Illia Silin
0b07559cbe Revert "Add ck tile examples to package (#1880)" (#2150) 
[ROCm/composable_kernel commit: 9a9f59ae69]
 2025-04-30 10:20:16 -07:00
Bartłomiej Kocot
a78cbfd6fc Add grouped conv fwd 16x16 mfma instruction instances (#2140) 
* Add grouped conv fwd 16x16 mfma instruction instances

* fix

* remove oddc

* fix

* fix

[ROCm/composable_kernel commit: 23de234dbe]
 2025-04-30 09:49:37 +02:00
Aviral Goel
b537ae86b9 [Tile Engine] Improved README.md (#2134) 
* improved tile_engine readme

* changed ck tile explanation and json

* further improved readme

* fixed typo

[ROCm/composable_kernel commit: 1aea51d34e]
 2025-04-29 17:37:07 -07:00
Max Podkorytov
a65374ed85 try building ck4inductor and testing it inside a virtual environment (#2142) 
use system virtualenv

use python-full ubuntu package in docker image

---------

Co-authored-by: illsilin <Illia.Silin@amd.com>
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>

[ROCm/composable_kernel commit: 6601931949]
 2025-04-29 17:22:38 -07:00
Illia Silin
8346274d24 Run CI jobs as user jenkins (#2141) 
* run CI as jenkins

* remove user jenkins from docker image

* move inductor installation to a writeable path

* add a switch for inductor tests

[ROCm/composable_kernel commit: 8fcb4dff1a]
 2025-04-29 07:35:10 -07:00
Aviral Goel
d79abc2e03 Add Matrix A and Matrix B Swizzle for LDS in Computev4 policy (#2136) 
* fixed computev4 policy bug for lds swizzle

* added swizzle for input matrix B

* Improved ComputeV4 policy and pipeline by swizzling A and B

* consolidated LDS descriptor functions in parent struct

[ROCm/composable_kernel commit: 65f182d617]
 2025-04-28 18:20:47 -07:00
Khushbu Agarwal
aeb46e6a49 Support for MFMA_16x16x128 for fp8/bf8 (#2125) 
* Adding 16x16x128 support for gfx950

* Support for fp8 and bf8

* fix input arguments for MFMA scale instruction

* clang-formatted

* Fixes for lwpck-3145 (#2138)

* Fix lds tile & cmake dep & default epilogue

* Fallback BTypeToUse to ADataType in WOQ cases

* reverting instance json file

* reverting instance json file

---------

Co-authored-by: Yi DING <yi.ding@amd.com>

[ROCm/composable_kernel commit: d107f3c3a5]
 2025-04-28 18:19:50 -07:00
Khushbu Agarwal
10188b5103 [TileEngine] Support for sparsity in codegen (#2128) 
* Added sparsity flag in codegen

* remove comments

* clan formatted

* added sparsity as runtime argument

* updated README

* updated stream config variable

* fix typo for tail_num in hot loop

[ROCm/composable_kernel commit: 768c99eca9]
 2025-04-28 18:19:23 -07:00
Bartłomiej Kocot
7942bb905b Integrate universal gemm with conv bwd data and add SplitK (#1315) 
* Integrate universal gemm with conv bwd data

* Fix multi d kernel

* Add splitK support

* instances refactor

* instances refactor

* refactor

* fixeS

* fixes

* 16x16 instnaces

* Fixes

* Fix

* Fix

* Fix

* Fix

* Fix

* Fixes

* fix

* fix

[ROCm/composable_kernel commit: 4094ad158a]
 2025-04-28 23:54:49 +02:00
Daniel Su
023137b227 Check max-ilp-scheduling compiler option for moe_gemm examples (#2127) 
[ROCm/composable_kernel commit: d9786f3363]
 2025-04-28 13:40:22 -07:00
jakpiase
28783ec2f4 Add ck tile examples to package (#1880) 
* add ck tile examples to package

* Update jenkinsfile

* fix for jenkinsfile

* fix for building ck tile code on non gfx9

* compile ck tile examples only for gfx94

* include ck tile examples in all target

* fix for basic gemm UseStructuredSparsity

* Update CMakeLists.txt

* Update gemm_pipeline_problem.hpp

* add targets to rocm install

---------

Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>

[ROCm/composable_kernel commit: 434d19f696]
 2025-04-28 09:53:19 -07:00
lalala-sh
cfa85fa761 fix moe i4 example bug (#2139) 
[ROCm/composable_kernel commit: 83394e40d2]
 2025-04-28 09:49:31 -07:00
Anton Gorenko
420de0f22d DeviceGemm_Wmma_CShuffleV3 with BlockGemmPipelineVersion::v3 (#2096) 
* Prepare files for DeviceGemm_Wmma_CShuffleV3

* Implement main part of CShuffleV3 with block pipeline v3 for WMMA

* Remove unused functions and template params for A/B descriptors

* Support both gfx11 and gfx12

* Enable SplitK for gfx12 and disable for gfx11

* Added RowColRow layout for DeviceGemmV2 fp16

* Added more instances for Row, Col, Row data layout

* Added instances for DeviceGemm_Wmma_CShuffleV3, Col, Row, Row data layout

* Added instances for DeviceGemm_Wmma_CShuffleV3, Col, Col, Row data layout

* Added more instances for DeviceGemm_Wmma_CShuffleV3, Row, Row, Row data layout

* Fix formatting

* Add documentation

Based on cc666c6a19dabc2cce8141e7ae23bd460ceef331

* Enable gemm_universal profiling for gfx11/12

* Add WMMA intrinsics for F8/BF8

* Support F8/BF8 DeviceGemm_Wmma_CShuffleV3, add basic instances

* Add BF16 instances and tests

* Fix test_gemm_universal_wmma_fp8 by adding CK_USE_WMMA_FP8

---------

Co-authored-by: Anca Hamuraru <anca@streamhpc.com>

[ROCm/composable_kernel commit: edd92fc546]
 2025-04-28 10:14:21 +05:00
Yi DING
77c7fb1e6b Fix fp8 convert & add option for basic example (#2129) 
[ROCm/composable_kernel commit: 8add2cf45d]
 2025-04-27 16:26:05 -07:00
Po Yen Chen
88d4c6baab Avoid using store_tile_raw() for fp32 tensors (#2072) 
[ROCm/composable_kernel commit: 3d4d70d2fc]
 2025-04-26 23:07:41 -07:00
joyeamd
32053c0b27 SWDEV-52596 for hdim=256, when use splitkv pipeline, two new pipelines need to be added (#2126) 
[ROCm/composable_kernel commit: 41541aff7a]
 2025-04-25 16:31:09 +08:00
Khushbu Agarwal
5c17305a82 MFMA_32x32x16 for gfx950 (#2121) 
* Enable MFMA_32x32x16 for fp16/BF16 for gfx950

* clang formatted

[ROCm/composable_kernel commit: a2ed34a112]
 2025-04-24 10:20:22 -07:00
Illia Silin
08f859b460 make code compliant with std=c++20 (#2123) 
[ROCm/composable_kernel commit: 01cb8379cd]
 2025-04-24 10:14:52 -07:00
alexxu-amd
441a7d9847 Setup Doxygen API reference for Docs (#2115) 
* setup Doxygen settings

* add api_reference to requirements.txt

* add doxygen file header

* omit latex generation

* remove testing entry

* update Doxyfile

[ROCm/composable_kernel commit: ba97363acd]
 2025-04-24 09:35:06 -06:00
rocking
8100d99c5d Only generate specific hdim (#2120) 
[ROCm/composable_kernel commit: 02ce6d39ea]
 2025-04-24 18:52:58 +08:00
carlushuang
65199be10e [CK_TILE] support gfx950 matrix core in 01_fmha fwd (#2110) 
* gfx950 01_fmha fwd

* fix comment

---------

Co-authored-by: Thomas Ning <Thomas.Ning@amd.com>

[ROCm/composable_kernel commit: 5487289fc4]
 2025-04-23 12:40:18 -07:00
John Afaganis
fd63cc6e10 Update CODEOWNERS (#2119) 
[ROCm/composable_kernel commit: 854159fd00]
 2025-04-23 10:25:41 -07:00
lalala-sh
8f426b1216 Moe gemm activation (#2026) 
* fix useless code and remove usless oob

* clang format

* fix coredump in e2e test

* fix2

* fix clang format

* fix output oob

* impl int64 but result not correct

* int64 index ok now

* input output all ok

* fix uint32

* revert v1 test

* use uint32

* mork to support 13w tokens

* moe sorting fix moebuf

* fix merge

* update moe api fix aiter build

* fix buid

* fuse silu

* silu ok

* acale ok

* add silu

* change code

* gemm2 ok

* gufusion compatible ok, fix warnings

* gu fusion for m32 m64 ok

* support bf16 cshuffle

* i4 gemm2 ok

* i4 gemm2 ok and i4 gemm1 build

* 16x16 run ok

* change flops; change cshuffle dtype

* fuse gelu silu act in moe gemm1

* fp8 with act ready

* int4 act ready

* remove useless changes

* remove useless code change

* fix clang format

* add the arch limit of int4 moe gemm

* fuse moe activation

* fix fp8 16x16

* fix no quant case

* fix bugs

* fix fp8 gufusion bug

* remove useless comments

* refine activation code & complete moe example

* fix int8 bugs

* merge tkw1

---------

Co-authored-by: coderfeli <coderfeli@163.com>
Co-authored-by: feli <felix.li@amd.com>
Co-authored-by: illsilin <Illia.Silin@amd.com>
Co-authored-by: root <root@hjbog-srdc-51.amd.com>
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>

[ROCm/composable_kernel commit: 39ba03f25d]
 2025-04-23 10:35:34 +08:00
Khushbu Agarwal
c6b7f48326 Adding include directory in tile_engine (#2116) 
[ROCm/composable_kernel commit: 94662b02d0]
 2025-04-22 15:55:19 -07:00
Gino Lu
983dac1699 [CK-Tile] warp-gemm support for using V_MFMA_F32_16x16x32_BF16 (#2073) 
* draft v_mfma_f32_16x16x32_bf16

* fix error config and add debug code.

* Solve the CShuffle Problem

* draft v_mfma_f32_16x16x32_bf16

* fix error config and add debug code.

* Solve the CShuffle Problem

* fix error while testing new command

* Finished the feature of new mfma 16*16*32

* Addressed the comment

---------

Co-authored-by: ThomasNing <thomas.ning@amd.com>

[ROCm/composable_kernel commit: 504f563f78]
 2025-04-22 15:52:36 -07:00
Rostyslav Geyyer
dbcaae42bf Temporarily disable MX FP4 device tests (#2112) 
[ROCm/composable_kernel commit: 416e851584]
 2025-04-22 16:08:48 -05:00
Thomas Ning
005e61ce63 GEMM Multiply Multiply Fix (#2102) 
* fix the type convert and increase the BF16 conversion + the profile comment

* fix the CI

[ROCm/composable_kernel commit: 0cca8fa28f]
 2025-04-22 01:13:22 -07:00
Thomas Ning
863ec4eb88 update code owner (#2113) 
[ROCm/composable_kernel commit: 4bef60aa57]
 2025-04-21 13:53:03 -07:00
Muhammed Emin Ozturk
eb8306b5f0 MI308 fix for streamk 1-Tile floating point exception (#2101) 
[ROCm/composable_kernel commit: b092c18da7]
 2025-04-21 11:44:07 -07:00
Thomas Ning
83197c769f MFMA 16x16x32fp8 (#2103) 
* add mfma_16x16x32_fp8

* clang format code

* Finished the fix for gemm basic

* clang foramt

* rebuild CI

* recover gemm.hpp

* add MFMA 16*16*32bf8

---------

Co-authored-by: solin <bingzhou@amd.com>

[ROCm/composable_kernel commit: a738e43445]
 2025-04-21 10:21:35 -07:00
Illia Silin
b29fc0efce fix daily gfx942 build (#2106) 
[ROCm/composable_kernel commit: ce61759538]
 2025-04-21 08:48:22 -07:00
Khushbu Agarwal
74210a9dfc multi instance generation for CkTileEngine (#2080) 
* Add support for multi-instance verification, print detail for each instance, documentation fix

* clang formatted

* Added Readme file

* updated readme

* Addressing review comments

* clang formatted

* Updated ReadMe and GPU reference code

* simplified dispatch kernel code

* indentation

[ROCm/composable_kernel commit: 7cadf187e2]
 2025-04-21 08:39:45 -07:00
solin
dd2c3289c9 fix CI build fail 
[ROCm/composable_kernel commit: c318ec0778]
 2025-04-21 16:00:12 +08:00
lalala-sh
dc58110b06 enable do top k weights in moe stage1 gemm (#2094) 
* add switch for mul topk weights

* fix bf16/f16 bugs

* complete

[ROCm/composable_kernel commit: bcf5bb41be]
 2025-04-18 10:45:49 +08:00
Andriy Roshchenko
7972a39081 MX GEMM - Parameterized Test Template (#2088) 
* Tests for MX FP8 GEMM

* Improve documentation


[ROCm/composable_kernel commit: 213b203a3c]
 2025-04-16 19:56:00 -06:00
Andriy Roshchenko
a96e96ab4e MX GEMM - Add MX BF8 example (#2071) 
* Add MX GEMM example for MX BF8

* Verified MX FP8 with 16x16x128 scale builtin

* Verify MX BF8 GEMM with BF16 output


[ROCm/composable_kernel commit: da54464cce]
 2025-04-16 15:25:02 -06:00
Illia Silin
729c668e9d Upgrade default docker to Ubuntu24.04 (#2090) 
* upgrade docker to Ubuntu24.04

* add break-system-packages flag to pip install

* fix dockerfile

[ROCm/composable_kernel commit: 3bb62f16cd]
 2025-04-16 12:10:15 -07:00
aledudek
d02e727b51 Add grouped conv fwd 3d GKCYX instances for f32, f16, bf16 (#2069) 
* Part1

* Add grouped conv fwd 3d GKCYX instances for f32, f16, bf16

* Add missing coma

* Add missing cpp instance files

* Fix 3d layout

* Add missing closing bracket

* Add missing comp x2 and part2 instances

* Fix typo in instance name

* fix

* Fix

---------

Co-authored-by: Bartlomiej Kocot <barkocot@amd.com>

[ROCm/composable_kernel commit: 7c32652e03]
 2025-04-16 11:00:55 +02:00
BingYuan.Zhou
f6b51a1c7b [flatmm] implement basic fp16 flatmm (#2089) 
* [flatmm] implement basic fp16 flatmm

* fix CI build fail

---------

Co-authored-by: root <root@hjbog-srdc-50.amd.com>
Co-authored-by: solin <bingzhou@amd.com>

[ROCm/composable_kernel commit: eaf1f0bf3b]
 2025-04-16 16:51:17 +08:00