ROCm/composable_kernel
1
0
Fork 0
mirror of https://github.com/ROCm/composable_kernel.git synced 2026-07-14 11:07:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,016 Commits 778 Branches 40 Tags
dbf5378a08f769de4cb194bef3733f756abe6dfd
Commit Graph

139 Commits

Author SHA1 Message Date
OscarXu
dbf5378a08 gemm2 perf optimized without HotLoopScheduler. HotLoopScheduler WIP 2025-06-19 03:53:15 -05:00
OscarXu
e91c75aef3 Optimize moe_bs_fp8 gemm1 v1 pipeline. Perf of buf2lds is better than buf2gpr now. 2025-06-12 03:20:51 -05:00
OscarXu
79f47b4611 Add buf2lds moe_bs_fp8 support to gemm1 v3 and gemm2 v1/v3. Add result_all_zero check 2025-06-11 03:06:57 -05:00
OscarXu
2938b13fee buf2lds moe_bs_fp8_gemm1 func pass 2025-06-09 03:16:35 -05:00
OscarXu
63053f49ed Fix moe blockscale gemm1 barrier 0x800 for new compiler 2025-05-29 23:11:40 -05:00
OscarXu
52d68c9529 flag and barrier fix for copmiler branch MainOpSelV3 2025-05-29 03:13:11 -05:00
OscarXu
772debdf8f Fix do_weight in gemm1. Fix cshuffle_datatype. Clang-format 2025-05-28 18:29:06 +08:00
OscarXu
9fdfff82ea Fix v2 topk_weight cal. Add silu asm. 2025-05-20 13:42:06 +08:00
OscarXu
215ddb0be0 Merge remote-tracking branch 'origin/moe_bs_stage1_dev' into moe_merge_v3_bs_for_aiter 2025-05-17 04:35:43 -05:00
OscarXu
9c9bf1aece Moe blockscale gemm1&gemm2 asm support for aiter. Suppression cmkae flag til new compler. 2025-05-17 04:33:52 -05:00
OscarXu
e4516b71c9 Merge branch 'moe_bs_stage1_dev' into moe_merge_v3_bs_for_aiter 2025-05-17 00:43:46 -05:00
OscarXu
98606fad94 Merge remote-tracking branch 'origin/wjx/moe_v3_aiter' into moe_bs_stage1_dev 2025-05-14 20:48:42 -05:00
OscarXu
c84ad83905 v1 fix for mi350. 2025-05-14 08:05:17 -05:00
valarLip
5f81710346 fix blockscale gemm bug 2025-05-14 08:23:49 +00:00
OscarXu
9ecbcce2e9 Merge remote-tracking branch 'origin/wjx/moe_v3_aiter' into moe_merge_v3_bs_for_aiter 2025-05-14 15:05:08 +08:00
OscarXu
b2c0c3cfeb gemm1 v3 64x128x128 debug 2025-05-14 10:07:11 +08:00
OscarXu
50590b8f17 Add gemm1 v1 32x128x128 2025-05-13 16:46:51 +08:00
valarLip
eaebefb278 update moe v1 pipeline 2025-05-13 08:42:49 +00:00
OscarXu
19aa39b978 Merge remote-tracking branch 'origin/wjx/moe_v3_aiter' into moe_bs_stage1_dev 2025-05-13 10:46:40 +08:00
OscarXu
f392874e3d revert unexpected file change 2025-05-13 09:55:33 +08:00
OscarXu
fe8bb251da gemm1 up-only pass. GU WIP 2025-05-12 21:08:42 +08:00
OscarXu
0a845c0b7d Merge remote-tracking branch 'origin/moe_blockscale_dev_gfx950' into moe_merge_v3_bs_for_aiter. Manual fix for thread_group_tensor_slice_transfer_v7r3_scatter 2025-05-12 19:25:39 +08:00
valarLip
3ea573acd6 update moe 2025-05-12 08:10:50 +00:00
valarLip
037a5ff76c update moe 2025-05-12 06:36:23 +00:00
lalala-sh
d9ab2fe4f5 update moe 2025-05-12 05:08:58 +00:00
OscarXu
3e5005a47b fix compile error. Gemm2 pass. Gemm1 WIP 2025-05-12 11:21:14 +08:00
OscarXu
7729fdb349 temp save for gemm1. Function not ready 2025-05-12 09:32:58 +08:00
OscarXu
03d5a50b45 Merge remote-tracking branch 'origin/wjx/align_v3_pipeline' into moe_bs_stage1_dev 2025-05-09 20:15:11 +08:00
lalala-sh
6c459e8c38 Merge branch 'develop' into wjx/align_v3_pipeline 2025-05-08 18:08:31 +08:00
lalala-sh
960b2bce1c update 2025-05-08 09:48:23 +00:00
OscarXu
586c2a9a15 Merge remote-tracking branch 'origin/f8blk_scale_opt' into moe_blockscale_dev 2025-05-08 16:39:08 +08:00
aska-0096
68c7fe843d temp save, a performant version. 2025-05-08 07:44:06 +00:00
OscarXu
62877fa074 16x16 function merged to moe 2025-05-08 14:33:31 +08:00
OscarXu
8021128572 merge origin/f8blk_scale_opt 2025-05-07 17:10:03 +08:00
lalala-sh
0ab978584d fix bugs 2025-05-06 07:36:59 +00:00
aska-0096
8c40f3d8fd experimental optimization for scale load in blkscale gemm 2025-05-02 07:12:01 +00:00
lalala-sh
9c06c3817a [fix] align v3 gufusion pipeline 2025-04-30 02:27:39 +00:00
lalala-sh
b8427b812e align v3 gufusion pipeline 2025-04-30 01:35:49 +00:00
aska-0096
240817cd49 blockscale preshuffle, pipeline optimized, compiler issue 2025-04-28 10:20:23 +00:00
Anton Gorenko
edd92fc546 DeviceGemm_Wmma_CShuffleV3 with BlockGemmPipelineVersion::v3 (#2096) 
* Prepare files for DeviceGemm_Wmma_CShuffleV3

* Implement main part of CShuffleV3 with block pipeline v3 for WMMA

* Remove unused functions and template params for A/B descriptors

* Support both gfx11 and gfx12

* Enable SplitK for gfx12 and disable for gfx11

* Added RowColRow layout for DeviceGemmV2 fp16

* Added more instances for Row, Col, Row data layout

* Added instances for DeviceGemm_Wmma_CShuffleV3, Col, Row, Row data layout

* Added instances for DeviceGemm_Wmma_CShuffleV3, Col, Col, Row data layout

* Added more instances for DeviceGemm_Wmma_CShuffleV3, Row, Row, Row data layout

* Fix formatting

* Add documentation

Based on e5ad48a784

* Enable gemm_universal profiling for gfx11/12

* Add WMMA intrinsics for F8/BF8

* Support F8/BF8 DeviceGemm_Wmma_CShuffleV3, add basic instances

* Add BF16 instances and tests

* Fix test_gemm_universal_wmma_fp8 by adding CK_USE_WMMA_FP8

---------

Co-authored-by: Anca Hamuraru <anca@streamhpc.com>
 2025-04-28 10:14:21 +05:00
aska-0096
d7d72543e8 Merge branch 'f8blockscale_bpreshuffle' of https://github.com/ROCm/composable_kernel into swdev_528812 2025-04-28 03:35:53 +00:00
aska-0096
bc9c819aa4 generalized bpreshuffle pipeline optimization 2025-04-27 11:50:30 +00:00
aska-0096
49338edb1b tempsave 2025-04-27 08:05:20 +00:00
aska-0096
946a2119cd temp save 2025-04-25 05:12:47 +00:00
coderfeli
cd7955ce8b Merge branch 'swdev_528812' into dev/moe_opt 2025-04-25 02:51:33 +00:00
coderfeli
ddb5f36eeb add missing file 2025-04-24 11:10:31 +00:00
coderfeli
ceaa5a984b gu fusion v3 2025-04-24 11:05:08 +00:00
aska-0096
d6e2dd92fe enable f8 new mfma for preshuffle gemm. found some vectorizer issue even with slp flag 2025-04-23 15:53:15 +00:00
lalala-sh
39ba03f25d Moe gemm activation (#2026) 
* fix useless code and remove usless oob

* clang format

* fix coredump in e2e test

* fix2

* fix clang format

* fix output oob

* impl int64 but result not correct

* int64 index ok now

* input output all ok

* fix uint32

* revert v1 test

* use uint32

* mork to support 13w tokens

* moe sorting fix moebuf

* fix merge

* update moe api fix aiter build

* fix buid

* fuse silu

* silu ok

* acale ok

* add silu

* change code

* gemm2 ok

* gufusion compatible ok, fix warnings

* gu fusion for m32 m64 ok

* support bf16 cshuffle

* i4 gemm2 ok

* i4 gemm2 ok and i4 gemm1 build

* 16x16 run ok

* change flops; change cshuffle dtype

* fuse gelu silu act in moe gemm1

* fp8 with act ready

* int4 act ready

* remove useless changes

* remove useless code change

* fix clang format

* add the arch limit of int4 moe gemm

* fuse moe activation

* fix fp8 16x16

* fix no quant case

* fix bugs

* fix fp8 gufusion bug

* remove useless comments

* refine activation code & complete moe example

* fix int8 bugs

* merge tkw1

---------

Co-authored-by: coderfeli <coderfeli@163.com>
Co-authored-by: feli <felix.li@amd.com>
Co-authored-by: illsilin <Illia.Silin@amd.com>
Co-authored-by: root <root@hjbog-srdc-51.amd.com>
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
 2025-04-23 10:35:34 +08:00
aska-0096
5366d3415b f8 mfma issue 2025-04-22 10:59:03 +00:00