Commit Graph

592 Commits

Author SHA1 Message Date
Qun Lin
40236bbb79 add instance factory + disable output lds 2025-06-11 14:55:05 +08:00
Lin, Qun
005d1c0b8c simplify code 2025-06-11 00:39:42 -05:00
Lin, Qun
88992a7856 fix fwd v1 quality issue. (28 x 5 x 1) 2025-06-11 00:07:49 -05:00
Qun Lin
169e4277b5 fwd v1 2025-06-10 17:00:26 +08:00
Qun Lin
01c1138bfe fwd 2025-06-08 19:23:03 +08:00
Lin, Qun
1f083ab8b5 correct display info 2025-06-07 02:24:30 -05:00
Qun Lin
397c9c9865 add new instance for tuning 2025-06-07 14:49:01 +08:00
Qun Lin
9b5f86cec6 fix k-split 2025-06-06 08:15:41 -05:00
Lin, Qun
badd82498a fix bugs 2 2025-06-06 04:31:39 -05:00
Lin, Qun
9b7260f544 fix bugs in wsplit and k-split 2025-06-05 23:37:23 -05:00
Qun Lin
7f93429797 add wsplit 2025-06-05 17:40:48 +08:00
Qun Lin
fca03f72fc support k-split 2025-06-05 13:05:49 +08:00
Lin, Qun
289fdf59e1 finalize code 2025-06-03 23:25:07 -05:00
Lin, Qun
a61d19ff8f fix a typo 2025-06-03 19:28:10 -05:00
Qun Lin
5d1c8c826f add pipelinv 2 2025-06-03 21:30:22 +08:00
Lin, Qun
4c39de8b8e fix bugs 2025-06-03 06:26:42 -05:00
Lin, Qun
3f18187dd9 fix bug 2
- kernel 1-3 passed
2025-06-03 02:21:31 -05:00
Lin, Qun
3516dbdf56 fix bugs. 2025-06-03 01:24:45 -05:00
Qun Lin
6d12067a30 fix a typo 2025-06-02 16:17:58 +08:00
Qun Lin
eb158d9b86 add device factory and fix build error. 2025-06-02 15:05:10 +08:00
Qun Lin
b27dddfb59 bwd weight dl v4 - v0 2025-06-02 10:39:13 +08:00
Qun Lin
044df51b56 add new class 2025-05-30 20:25:44 +08:00
Lin, Qun
75b6cd4934 fix quality issue
option MergeShareMem  doesn't work when Num_Wave > 1
2025-05-29 23:28:37 -05:00
Lin, Qun
26b8a1d972 change layout to gnhwc 2025-05-29 19:28:38 -05:00
Lin, Qun
321dc5dd39 version 3 2025-05-29 08:25:18 -05:00
Lin, Qun
5fd4aef2dc version 2 2025-05-29 07:38:04 -05:00
Lin, Qun
0750ebb13d version 1 2025-05-29 01:08:04 -05:00
Qun Lin
b9466aff4d tmp code 2 2025-05-29 09:01:03 +08:00
Qun Lin
45b4c48000 tmp code 1 2025-05-28 15:47:05 +08:00
Lin, Qun
2e57e6e61c refine example_conv_bwd_weight 2025-05-27 08:54:35 +08:00
Lin, Qun
f5377edfe5 refine test example_grouped_conv_bwd_weight_xdl_fp16 2025-05-27 08:54:35 +08:00
Lin, Qun
94ae9ae73f update ck_tile script 2025-05-27 08:54:33 +08:00
Lin, Qun
fb212bd921 fix ck tile build error 2025-05-27 08:53:50 +08:00
Qun Lin
027d58110a disable test to pass compile. 2025-05-27 08:52:30 +08:00
Qun Lin
acf28d2c41 rebase flatmm fp8 2025-05-27 08:52:28 +08:00
Qun Lin
feeb6f7cd8 refine universal gemm multi-config 2025-05-27 08:49:57 +08:00
Qun Lin
512e0d7935 fix test scripts 2025-05-27 08:49:57 +08:00
Qun Lin
547c602d85 support multi config in universal gemm 2025-05-27 08:49:57 +08:00
Lin, Qun
16bb17dc71 add config 7 2025-05-27 08:49:57 +08:00
Lin, Qun
ede9f6ca8b add config 3.4.5.6 2025-05-27 08:49:57 +08:00
Qun Lin
efe964bd95 1. copy bhalf v3 to half v3
2. add multi config in half v2
3. sync block universal with half v3
2025-05-27 08:49:54 +08:00
Zzz9990
ece38b9d7a [VLLM V1] Add chunked prefill for FA to pass seq with small seqlen_q (#2221)
* fix splitkv compiler issue since lse is used to select kernel instances

* bypass seqlen == 1

* add chunked prefill into mha varlen

This reverts commit aa9847e42d.

* skip compile when receipt 2-4 and add comments

* fix

---------

Co-authored-by: fsx950223 <fsx950223@outlook.com>
2025-05-26 19:17:18 +08:00
Illia Silin
8146e471f1 fix the buffer intrinsic names for clang >=20 (#2228) 2025-05-23 14:58:25 -07:00
Illia Silin
1b846143c6 Revert "Update the buffer load/store intrinsic names for clang>=20. (#2192)" (#2227)
This reverts commit 58f9e9ffbc.
2025-05-22 15:41:17 -07:00
SamiAario-AMD
380bca2b85 Fix 11_add_rmsnorm2d_rdquant (#2207) 2025-05-20 15:15:28 -07:00
Thomas Ning
1386924749 Add the instances for small sized GEMM in preshuffle and improve CMake Flag (#2212)
* Add small instance, add the bug fix, & improve the example CMake

* clang format
2025-05-20 15:05:08 -07:00
Sami Remes
d1e6f0982d [CK_TILE] Grouped GEMM tile loop (#2146)
* Add trait to use a persistent kernel and split the entrypoints in grouped gemm

* Some helper functions for persistent kernel case

* Get max occupancy grid using device properties

* Implement tile loop in main entry point to grouped gemm

* Enable GridSize() on device

* Handle offset tile index using real current block index

* Add persistent kernel choice to grouped gemm example

* Use a for-loop for iterating over the group

* Reduce VGPR spills by early-exit

* Enable persistent kernel choice in grouped_gemm example

* Add persistent kernel option to grouped_gemm test

* Fix formatting with remod.py

* Remove GridUpdateBlocks as blocks are now iteratively computed

* Add comment about VGPR spilling

* Fix formatting

* Use CK_TILE_HOST instead of __host__

* Enable all Row/Col combinations in grouped gemm unit test

* Add some KBatch=2 cases to grouped gemm tests

* Fix SplitK for grouped gemm

* Enable pipeline hotloop/tailnumber selection in-kernel for grouped gemm

* Add type traits

* Split examples to regular and tileloop

* Formatting

* Use hipExtStreamGetCUMask to get current active CUs for the given stream

* Align test and example kernel config, and disable validation for splitk repeats

* Remove debug options from CMakeLists.txt

* Separate the code paths for persistent/non-persistent in test

* Fix formatting

* Address review comments

---------

Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>
2025-05-20 17:18:57 +03:00
Aviral Goel
c4929225f6 remove debug statements from CMakeLists (#2204) 2025-05-19 17:31:04 -07:00
jefyang1
f18170064d Use new mfma instructions for FP8 on gfx950 (#2202)
* Add logic to use new mfma instructions for fp8 bf8

* Fix example_gemm_xdl_fp8_pk_i4_bpreshuffle_v3 on gfx950 and run clang format

* Update include/ck/tensor_operation/gpu/warp/xdlops_gemm.hpp

Co-authored-by: Andriy Roshchenko <107577548+andriy-ca@users.noreply.github.com>

* Fix intrin_mfma f8 calls due to merge mistake

---------

Co-authored-by: Andriy Roshchenko <107577548+andriy-ca@users.noreply.github.com>
2025-05-19 17:29:51 -07:00
jefyang1
b8b12bb81e Fix example_grouped_gemm_multiple_d_xdl_fp16 on gfx950 (#2203)
* Fix example_grouped_gemm_multiple_d_xdl_fp16 on gfx950

* Run clang format
2025-05-19 14:25:50 -07:00