ROCm/composable_kernel
1
0
Fork 0
mirror of https://github.com/ROCm/composable_kernel.git synced 2026-07-15 03:30:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,969 Commits 779 Branches 40 Tags
169e4277b5d006de297dcfb0e0eeca13dd2548b7
Commit Graph

589 Commits

Author SHA1 Message Date
Qun Lin
169e4277b5 fwd v1 2025-06-10 17:00:26 +08:00
Qun Lin
01c1138bfe fwd 2025-06-08 19:23:03 +08:00
Lin, Qun
1f083ab8b5 correct display info 2025-06-07 02:24:30 -05:00
Qun Lin
397c9c9865 add new instance for tuning 2025-06-07 14:49:01 +08:00
Qun Lin
9b5f86cec6 fix k-split 2025-06-06 08:15:41 -05:00
Lin, Qun
badd82498a fix bugs 2 2025-06-06 04:31:39 -05:00
Lin, Qun
9b7260f544 fix bugs in wsplit and k-split 2025-06-05 23:37:23 -05:00
Qun Lin
7f93429797 add wsplit 2025-06-05 17:40:48 +08:00
Qun Lin
fca03f72fc support k-split 2025-06-05 13:05:49 +08:00
Lin, Qun
289fdf59e1 finalize code 2025-06-03 23:25:07 -05:00
Lin, Qun
a61d19ff8f fix a typo 2025-06-03 19:28:10 -05:00
Qun Lin
5d1c8c826f add pipelinv 2 2025-06-03 21:30:22 +08:00
Lin, Qun
4c39de8b8e fix bugs 2025-06-03 06:26:42 -05:00
Lin, Qun
3f18187dd9 fix bug 2 
- kernel 1-3 passed
 2025-06-03 02:21:31 -05:00
Lin, Qun
3516dbdf56 fix bugs. 2025-06-03 01:24:45 -05:00
Qun Lin
6d12067a30 fix a typo 2025-06-02 16:17:58 +08:00
Qun Lin
eb158d9b86 add device factory and fix build error. 2025-06-02 15:05:10 +08:00
Qun Lin
b27dddfb59 bwd weight dl v4 - v0 2025-06-02 10:39:13 +08:00
Qun Lin
044df51b56 add new class 2025-05-30 20:25:44 +08:00
Lin, Qun
75b6cd4934 fix quality issue 
option MergeShareMem  doesn't work when Num_Wave > 1
 2025-05-29 23:28:37 -05:00
Lin, Qun
26b8a1d972 change layout to gnhwc 2025-05-29 19:28:38 -05:00
Lin, Qun
321dc5dd39 version 3 2025-05-29 08:25:18 -05:00
Lin, Qun
5fd4aef2dc version 2 2025-05-29 07:38:04 -05:00
Lin, Qun
0750ebb13d version 1 2025-05-29 01:08:04 -05:00
Qun Lin
b9466aff4d tmp code 2 2025-05-29 09:01:03 +08:00
Qun Lin
45b4c48000 tmp code 1 2025-05-28 15:47:05 +08:00
Lin, Qun
2e57e6e61c refine example_conv_bwd_weight 2025-05-27 08:54:35 +08:00
Lin, Qun
f5377edfe5 refine test example_grouped_conv_bwd_weight_xdl_fp16 2025-05-27 08:54:35 +08:00
Lin, Qun
94ae9ae73f update ck_tile script 2025-05-27 08:54:33 +08:00
Lin, Qun
fb212bd921 fix ck tile build error 2025-05-27 08:53:50 +08:00
Qun Lin
027d58110a disable test to pass compile. 2025-05-27 08:52:30 +08:00
Qun Lin
acf28d2c41 rebase flatmm fp8 2025-05-27 08:52:28 +08:00
Qun Lin
feeb6f7cd8 refine universal gemm multi-config 2025-05-27 08:49:57 +08:00
Qun Lin
512e0d7935 fix test scripts 2025-05-27 08:49:57 +08:00
Qun Lin
547c602d85 support multi config in universal gemm 2025-05-27 08:49:57 +08:00
Lin, Qun
16bb17dc71 add config 7 2025-05-27 08:49:57 +08:00
Lin, Qun
ede9f6ca8b add config 3.4.5.6 2025-05-27 08:49:57 +08:00
Qun Lin
efe964bd95 1. copy bhalf v3 to half v3 
2. add multi config in half v2
3. sync block universal with half v3
 2025-05-27 08:49:54 +08:00
Zzz9990
ece38b9d7a [VLLM V1] Add chunked prefill for FA to pass seq with small seqlen_q (#2221) 
* fix splitkv compiler issue since lse is used to select kernel instances

* bypass seqlen == 1

* add chunked prefill into mha varlen

This reverts commit aa9847e42d.

* skip compile when receipt 2-4 and add comments

* fix

---------

Co-authored-by: fsx950223 <fsx950223@outlook.com>
 2025-05-26 19:17:18 +08:00
Illia Silin
8146e471f1 fix the buffer intrinsic names for clang >=20 (#2228) 2025-05-23 14:58:25 -07:00
Illia Silin
1b846143c6 Revert "Update the buffer load/store intrinsic names for clang>=20. (#2192)" (#2227) 
This reverts commit 58f9e9ffbc.
 2025-05-22 15:41:17 -07:00
SamiAario-AMD
380bca2b85 Fix 11_add_rmsnorm2d_rdquant (#2207) 2025-05-20 15:15:28 -07:00
Thomas Ning
1386924749 Add the instances for small sized GEMM in preshuffle and improve CMake Flag (#2212) 
* Add small instance, add the bug fix, & improve the example CMake

* clang format
 2025-05-20 15:05:08 -07:00
Sami Remes
d1e6f0982d [CK_TILE] Grouped GEMM tile loop (#2146) 
* Add trait to use a persistent kernel and split the entrypoints in grouped gemm

* Some helper functions for persistent kernel case

* Get max occupancy grid using device properties

* Implement tile loop in main entry point to grouped gemm

* Enable GridSize() on device

* Handle offset tile index using real current block index

* Add persistent kernel choice to grouped gemm example

* Use a for-loop for iterating over the group

* Reduce VGPR spills by early-exit

* Enable persistent kernel choice in grouped_gemm example

* Add persistent kernel option to grouped_gemm test

* Fix formatting with remod.py

* Remove GridUpdateBlocks as blocks are now iteratively computed

* Add comment about VGPR spilling

* Fix formatting

* Use CK_TILE_HOST instead of __host__

* Enable all Row/Col combinations in grouped gemm unit test

* Add some KBatch=2 cases to grouped gemm tests

* Fix SplitK for grouped gemm

* Enable pipeline hotloop/tailnumber selection in-kernel for grouped gemm

* Add type traits

* Split examples to regular and tileloop

* Formatting

* Use hipExtStreamGetCUMask to get current active CUs for the given stream

* Align test and example kernel config, and disable validation for splitk repeats

* Remove debug options from CMakeLists.txt

* Separate the code paths for persistent/non-persistent in test

* Fix formatting

* Address review comments

---------

Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>
 2025-05-20 17:18:57 +03:00
Aviral Goel
c4929225f6 remove debug statements from CMakeLists (#2204) 2025-05-19 17:31:04 -07:00
jefyang1
f18170064d Use new mfma instructions for FP8 on gfx950 (#2202) 
* Add logic to use new mfma instructions for fp8 bf8

* Fix example_gemm_xdl_fp8_pk_i4_bpreshuffle_v3 on gfx950 and run clang format

* Update include/ck/tensor_operation/gpu/warp/xdlops_gemm.hpp

Co-authored-by: Andriy Roshchenko <107577548+andriy-ca@users.noreply.github.com>

* Fix intrin_mfma f8 calls due to merge mistake

---------

Co-authored-by: Andriy Roshchenko <107577548+andriy-ca@users.noreply.github.com>
 2025-05-19 17:29:51 -07:00
jefyang1
b8b12bb81e Fix example_grouped_gemm_multiple_d_xdl_fp16 on gfx950 (#2203) 
* Fix example_grouped_gemm_multiple_d_xdl_fp16 on gfx950

* Run clang format
 2025-05-19 14:25:50 -07:00
Po Yen Chen
8cb0474b3d Use only qr_async pipeline for batch_prefill (#2195) 2025-05-15 11:47:29 -07:00
Illia Silin
58f9e9ffbc Update the buffer load/store intrinsic names for clang>=20. (#2192) 
* fix the buffer load/store intrinsic names

* fix clang format
 2025-05-13 10:18:14 -07:00
Po Yen Chen
2920604786 [CK_TILE] Add logits soft-capping & customization support to the FMHA forward kernel/pipelines (#2163) 
* hack for cap logits

* fix bug

* Re-format files

* Allow specifying logits_soft_cap through APIs

* Support turn on/off logits_soft_cap in async pipeline

* Do not generate non-verified kernels

* Align receipt used in Aiter

* Sync logits soft-capping across pipelines

* Re-enable some hdim pipelines

* fix perf

* Add attention variant for logits_soft_cap

* Add newline at end-of-file

* Fix performance

* Add comment to explain logits_soft_cap pre-processing

* Unify code

* Unify floating-point literal style

* Use class data member to slience the compilation error

* [CK_TILE] Update attention customizaton interface: add LogitsMask() (#2133)

* Send 'mask' along with variant params to the LogitsMask()

* Send block indices to the variant

* Add indices parameters in variant interface

* Fix fmha bwd codegen error

* Allow switch logits_soft_cap impl

* Eliminate register spills

* Fix compilation errors

* Fix wrong LSE

* Fix LSE for splitkv kernel

* Sync splitkv pipeline changes

* Add batch_prefill kernel/pipeline

* Fix codegen error

* Undo changes in CMakeLists.txt

* Merge pipeline filtering check

* Use different code path if kHasLogitsSoftCap=false

* Remove [[maybe_unused]] attribute

* Use pre-existing compile-time flag to instantiate templates

* Sync pipeline changes

* Update CHANGELOG.md

---------

Co-authored-by: Bernard <bernaliu@amd.com>
Co-authored-by: coderfeli <coderfeli@163.com>
 2025-05-13 12:19:25 +08:00