ROCm/composable_kernel
1
0
Fork 0
mirror of https://github.com/ROCm/composable_kernel.git synced 2026-07-13 18:51:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,016 Commits 778 Branches 40 Tags
dbf5378a08f769de4cb194bef3733f756abe6dfd
Commit Graph

630 Commits

Author SHA1 Message Date
lalala-sh
abff33eaab tune fp8 example 2025-05-06 08:46:38 +00:00
OscarXu
e7fe8587f6 Add asm for no-loop v3_128x128x128 2025-05-06 10:47:29 +08:00
jakpiase
0bcb804ad0 [CK_TILE] Remove scratch usage from universal gemm (#2001) 
* moves kbatch condition outside of kernel

* add reviewer comments

* fixes

* fix tests

* fixes after review

---------

Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>
 2025-05-05 18:46:44 +02:00
aska-0096
8c40f3d8fd experimental optimization for scale load in blkscale gemm 2025-05-02 07:12:01 +00:00
Andriy Roshchenko
79b0bfeb41 MX GEMM - Add FP8 GEMM Tests for Different Layouts (#2152) 
* Add gemm_mx_fp8_bf8 example with row-major B

* Add more overloads of MX MFMA instructions

* Add MK_KN (RRR) tests

* Add KM_NK (CCR) tests

* Add more problem sizes to Large tests

* Add test_gemm_mx to the list of regression tests
 2025-05-01 11:55:48 -06:00
Illia Silin
9a9f59ae69 Revert "Add ck tile examples to package (#1880)" (#2150) 2025-04-30 10:20:16 -07:00
OscarXu
efbb85be2a Add .co direct asm support by CK_USE_ASM_MOE_STAGE2_BLOCKSCALE 2025-04-30 22:05:54 +08:00
lalala-sh
b8427b812e align v3 gufusion pipeline 2025-04-30 01:35:49 +00:00
Daniel Su
d9786f3363 Check max-ilp-scheduling compiler option for moe_gemm examples (#2127) 2025-04-28 13:40:22 -07:00
jakpiase
434d19f696 Add ck tile examples to package (#1880) 
* add ck tile examples to package

* Update jenkinsfile

* fix for jenkinsfile

* fix for building ck tile code on non gfx9

* compile ck tile examples only for gfx94

* include ck tile examples in all target

* fix for basic gemm UseStructuredSparsity

* Update CMakeLists.txt

* Update gemm_pipeline_problem.hpp

* add targets to rocm install

---------

Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
 2025-04-28 09:53:19 -07:00
lalala-sh
83394e40d2 fix moe i4 example bug (#2139) 2025-04-28 09:49:31 -07:00
aska-0096
240817cd49 blockscale preshuffle, pipeline optimized, compiler issue 2025-04-28 10:20:23 +00:00
OscarXu
f3fedde7e6 Fix int overflow casued by large B matrix 2025-04-28 15:55:10 +08:00
aska-0096
d7d72543e8 Merge branch 'f8blockscale_bpreshuffle' of https://github.com/ROCm/composable_kernel into swdev_528812 2025-04-28 03:35:53 +00:00
Yi DING
8add2cf45d Fix fp8 convert & add option for basic example (#2129) 2025-04-27 16:26:05 -07:00
aska-0096
bc9c819aa4 generalized bpreshuffle pipeline optimization 2025-04-27 11:50:30 +00:00
aska-0096
49338edb1b tempsave 2025-04-27 08:05:20 +00:00
Po Yen Chen
3d4d70d2fc Avoid using store_tile_raw() for fp32 tensors (#2072) 2025-04-26 23:07:41 -07:00
joyeamd
41541aff7a SWDEV-52596 for hdim=256, when use splitkv pipeline, two new pipelines need to be added (#2126) 2025-04-25 16:31:09 +08:00
aska-0096
1637dd5297 Merge branch 'swdev_528812_moe' of https://github.com/ROCm/composable_kernel into swdev_528812 2025-04-25 05:14:01 +00:00
aska-0096
946a2119cd temp save 2025-04-25 05:12:47 +00:00
coderfeli
e07ed1eda8 use v3 2025-04-25 03:15:48 +00:00
coderfeli
f9c29b5ec7 set 16x16 2025-04-25 03:09:53 +00:00
coderfeli
cd7955ce8b Merge branch 'swdev_528812' into dev/moe_opt 2025-04-25 02:51:33 +00:00
coderfeli
c3c4a1e252 change test 2025-04-24 11:09:41 +00:00
coderfeli
ceaa5a984b gu fusion v3 2025-04-24 11:05:08 +00:00
rocking
02ce6d39ea Only generate specific hdim (#2120) 2025-04-24 18:52:58 +08:00
coderfeli
2054e165bc fix moe pipeline and change to compute tile 2025-04-24 06:26:12 +00:00
aska-0096
abd40d3569 found a case that seems like have vectorizer issue 2025-04-23 15:55:59 +00:00
aska-0096
d6e2dd92fe enable f8 new mfma for preshuffle gemm. found some vectorizer issue even with slp flag 2025-04-23 15:53:15 +00:00
lalala-sh
39ba03f25d Moe gemm activation (#2026) 
* fix useless code and remove usless oob

* clang format

* fix coredump in e2e test

* fix2

* fix clang format

* fix output oob

* impl int64 but result not correct

* int64 index ok now

* input output all ok

* fix uint32

* revert v1 test

* use uint32

* mork to support 13w tokens

* moe sorting fix moebuf

* fix merge

* update moe api fix aiter build

* fix buid

* fuse silu

* silu ok

* acale ok

* add silu

* change code

* gemm2 ok

* gufusion compatible ok, fix warnings

* gu fusion for m32 m64 ok

* support bf16 cshuffle

* i4 gemm2 ok

* i4 gemm2 ok and i4 gemm1 build

* 16x16 run ok

* change flops; change cshuffle dtype

* fuse gelu silu act in moe gemm1

* fp8 with act ready

* int4 act ready

* remove useless changes

* remove useless code change

* fix clang format

* add the arch limit of int4 moe gemm

* fuse moe activation

* fix fp8 16x16

* fix no quant case

* fix bugs

* fix fp8 gufusion bug

* remove useless comments

* refine activation code & complete moe example

* fix int8 bugs

* merge tkw1

---------

Co-authored-by: coderfeli <coderfeli@163.com>
Co-authored-by: feli <felix.li@amd.com>
Co-authored-by: illsilin <Illia.Silin@amd.com>
Co-authored-by: root <root@hjbog-srdc-51.amd.com>
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
 2025-04-23 10:35:34 +08:00
aska-0096
25bb0d2fee add flags to avoid vectorizer problem 2025-04-23 02:08:44 +00:00
Gino Lu
504f563f78 [CK-Tile] warp-gemm support for using V_MFMA_F32_16x16x32_BF16 (#2073) 
* draft v_mfma_f32_16x16x32_bf16

* fix error config and add debug code.

* Solve the CShuffle Problem

* draft v_mfma_f32_16x16x32_bf16

* fix error config and add debug code.

* Solve the CShuffle Problem

* fix error while testing new command

* Finished the feature of new mfma 16*16*32

* Addressed the comment

---------

Co-authored-by: ThomasNing <thomas.ning@amd.com>
 2025-04-22 15:52:36 -07:00
aska-0096
5366d3415b f8 mfma issue 2025-04-22 10:59:03 +00:00
OscarXu
e61179e734 Add v3 support. Add compile_option 2025-04-22 16:42:33 +08:00
mtgu0705
578945b346 fix a bug, function passed. 2025-04-22 14:35:53 +08:00
mtgu0705
d6b84c7b40 fix xdl transpose. 2025-04-22 09:46:37 +08:00
mtgu0705
2f6529dcc2 updated 2025-04-21 10:51:37 +08:00
mtgu0705
066f209640 added moe blockscale, compiler successful, function not correct. 2025-04-19 16:02:51 +08:00
aska-0096
d883056744 Merge branch 'develop' of https://github.com/ROCm/composable_kernel into f8blockscale_bpreshuffle 2025-04-18 06:35:59 +00:00
aska-0096
7613cd60d6 temp save 2025-04-18 06:32:25 +00:00
lalala-sh
bcf5bb41be enable do top k weights in moe stage1 gemm (#2094) 
* add switch for mul topk weights

* fix bf16/f16 bugs

* complete
 2025-04-18 10:45:49 +08:00
mtgu0705
ef0b5f72af added moe_blockscale_dev branch to delete the per-token scale code. 2025-04-17 14:16:07 +08:00
Andriy Roshchenko
da54464cce MX GEMM - Add MX BF8 example (#2071) 
* Add MX GEMM example for MX BF8

* Verified MX FP8 with 16x16x128 scale builtin

* Verify MX BF8 GEMM with BF16 output
 2025-04-16 15:25:02 -06:00
BingYuan.Zhou
eaf1f0bf3b [flatmm] implement basic fp16 flatmm (#2089) 
* [flatmm] implement basic fp16 flatmm

* fix CI build fail

---------

Co-authored-by: root <root@hjbog-srdc-50.amd.com>
Co-authored-by: solin <bingzhou@amd.com>
 2025-04-16 16:51:17 +08:00
felix
c5975529bb add preshuffle gemm fp16 (#2036) 
* add preshuffle gemm fp16

* clang format and test ok

* Update gemm_multiply_multiply_xdl_fp16_bpreshuffle.cpp

remove useless comments in example

* Update gemm_multiply_multiply_xdl_fp16_bpreshuffle.cpp

remove 2

---------

Co-authored-by: coderfeli <coderfeli@163.com>
 2025-04-16 10:53:21 +08:00
joyeamd
94d47b1680 fmha hdim256 vectorize improve (#2086) 
For hdim 256, will not have vectorized buffer load when seqlen % 256 != 0 and hdim % 256 = 0; this commit tries to solve this condition.
 2025-04-16 09:21:04 +08:00
Andriy Roshchenko
7106976a72 MX GEMM - New GEMM pipeline for MX data types (#2059) 
* Allow selection of mfma_scale instructions

* Read B tensor from LDS to VGPR in chunks of 16 in MFMA order

* Add constexpr and synchronize return type for `get_exponent_value`

* Pass scales by reference and add comments to `mfma_scale_f32_32x32x64`

* Add support for microscaling instructions in `XdlopsGemm`

* Fix `mfma_scale_f32_16x16x128f8f6f4` wrapper

* Remove software implementation of MX GEMM

* Make interface of `intrin_mfma_scale_f32_16x16x128f8f6f4<16, 16>` consistent with the other scale instruction

* Update README

* Updated CHANGELOG

* Remove unused static methods
 2025-04-15 17:17:07 -06:00
Mingtao Gu
56378f810f CK pk_i4_t test failures fix (SWDEV-518629) (#2075) 
* fix pk_i4_v3 tests failures in Unbuntu env.

* fix pk_i4_t tests failure on Unbuntu issues.

* some fixed.

---------

Co-authored-by: mtgu0705 <mtgu@amd.com>
 2025-04-14 16:58:57 +08:00
mtgu0705
4dd4e8ed38 add example for moe stage2 fp8 blockscale 2025-04-14 10:50:28 +08:00