Commit Graph

489 Commits

Author SHA1 Message Date
Ding, Yi
e4a40c7214 Add fp8 profiler instances 2025-05-29 08:19:31 +00:00
Ding, Yi
aecac410d0 Merge remote-tracking branch 'origin/f8blk_scale_opt' into wip-f4-mergemoe-2 2025-05-28 11:15:22 +00:00
Ding, Yi
ad7fd89c1d Merge remote-tracking branch 'origin/feiw/mxfp4_moe_2Stages' into wip-f4 2025-05-28 09:28:26 +00:00
Ding, Yi
4f9bfb1566 Add more fp4 wp instances 2025-05-28 07:33:40 +00:00
Ding, Yi
857ef9f8c4 Merge preshuffle device 2025-05-28 07:02:28 +00:00
Ding, Yi
e2e0e0025e Profiler add f4 wp 2025-05-28 05:12:39 +00:00
Ding, Yi
b99c50a5d5 pad ascale 2025-05-28 03:35:33 +00:00
Ding, Yi
cf5b4c11a2 Pad shuffled a scale only 2025-05-28 02:37:14 +00:00
Ding, Yi
91eb136937 Fix v1; use M padding 2025-05-26 10:32:26 +00:00
Andriy Roshchenko
fdfc9c6fd8 Merge remote-tracking branch 'origin/develop' into andriy/wip-f4 2025-05-23 23:02:43 +00:00
Andriy Roshchenko
f03da29b65 Merge branch origin/wip-f4 into andriy/wip-f4 2025-05-23 22:14:30 +00:00
Illia Silin
bc2551ac3b disable building device_mha_operations by default (#2225) 2025-05-22 14:03:04 -07:00
Adam Dickin
417a6b65b6 Add MIOPEN_REQ_LIBS_ONLY option for cmake to build only the libs MIOpen requires (#2224)
* cut out anything we dont need for MIOpen to test

* refactor exclusion code to be more streamlined.
2025-05-22 11:14:33 -07:00
Ding, Yi
ce50d4bd62 Fix fp4 ckProfiler 2025-05-22 09:39:22 +00:00
Andriy Roshchenko
e302ab8f0c Merge branch origin/develop into wip-fp4 2025-05-22 06:31:47 +00:00
Bartłomiej Kocot
ebc5a6ef87 Grouped conv bwd wei add for larger filter and Merge Groupes optimization (#2197)
* Grouped conv bwd wei add two stage instances for larger filter and Merge Groups

* Fix

* fix

* Restore removed instances

---------

Co-authored-by: Rostyslav Geyyer <46627076+geyyer@users.noreply.github.com>
2025-05-21 22:47:34 +02:00
Thomas Ning
1386924749 Add the instances for small sized GEMM in preshuffle and improve CMake Flag (#2212)
* Add small instance, add the bug fix, & improve the example CMake

* clang format
2025-05-20 15:05:08 -07:00
Ding, Yi
667a356cc3 Add mx fp4 pileline v1 instances 2025-05-20 03:28:24 +00:00
Andriy Roshchenko
57e0f5df29 MX GEMM - Expand MX MFMA Testing to BF8, FP6, and BF6 Data Types (#2199)
* Unify test interface for different layouts.

* WIP: Introducing FP4/FP6/FP8 abstractions

* WIP: Introducing packed storage abstraction

* WIP: Introducing packed storage abstraction

* WIP: Improved support for FP6 data type

* Refactor packed storage for f6_t

* WIP: FP6 MFMA test

* Test if we correctly represent all FP6/FP4 numbers

* Additional output for failed FP4 test.

* More failing conversion tests

* Even more failing conversion tests

* Working FP6 MFMA tests

* Expand MX MFMA testing to BF8/6

* Update and verify MX MFMA test for packed types

* Fix fp4 and fp6 conversions on host

* Working MX MFMA tests for FP8/6/4

* Cleanup

* Add missing type

* Cleanup

* Final cleanup

* Restrict FP6/4 values output to CK_LOGGING=1

* Use CHAR_BIT instead of number 8

* Fix typo

* Remove FP6 and FP4 from the list of native types

---------

Co-authored-by: Rostyslav Geyyer <rosty.geyyer@amd.com>
2025-05-19 16:52:51 -05:00
Ding, Yi
ec240f391a Fix mx f4 ckProfiler 2025-05-19 06:39:43 +00:00
mtgu0705
a4b5a374b9 Merge remote-tracking branch 'origin/wip-f4-pk' into mx_moe_f4_scale_shuffle 2025-05-17 09:49:24 -05:00
Bartłomiej Kocot
6342f6b5e8 Restore oddc instances (#2201) 2025-05-16 18:42:02 -07:00
Illia Silin
40668c9a99 Build and store CK library deb package for all targets daily. (#2196)
* generate and store library package for all targets

* use ninja to build packages for all targets

* make sure to use ftime-trace when using ninja

* make sure build trace only runs on gfx9

* archive lib package and stash only library package
2025-05-16 07:40:53 -07:00
OscarXu
ec8d00d58d mx_moe_fp4 ready for aiter with clang-format. 2025-05-16 04:09:26 -05:00
OscarXu
c5be9a501b v1 function pass. 2025-05-16 03:16:38 -05:00
Ding, Yi
70e0d94932 Add f4 profiler examples 2025-05-16 07:49:55 +00:00
Ding, Yi
dc30e7d025 Add f4 ckProfiler 2025-05-16 07:19:22 +00:00
Ding, Yi
c04d44b5f6 Merge remote-tracking branch 'origin/develop' into wip-f4 2025-05-16 07:11:26 +00:00
Bartłomiej Kocot
7c0e29cc0f Extend 64x64 with 4 waves instances for grouped conv bwd wei (#2187)
* Extend 64x64 with 4 waves instnaces for grouped conv bwd wei

* Fix

* fix

* fix
2025-05-15 16:21:34 +02:00
OscarXu
f70f778e27 v1 compile pass. Function not ready 2025-05-15 08:01:56 -05:00
OscarXu
68dbe558df compile error fix 2025-05-15 16:55:20 +08:00
OscarXu
17922821ec Add gemm1 v1 2025-05-15 16:11:43 +08:00
mtgu0705
102151ebcf temp save 2025-05-14 08:13:47 -05:00
aska-0096
74686bf008 tempsave. Almost all instances passed. 2025-05-14 10:15:59 +00:00
feifei14119
7be8730247 fix cpu ref 2025-05-14 10:22:18 +08:00
mtgu0705
6dfe24c53e updated 2025-05-13 04:15:53 -05:00
mtgu0705
5ba86c210b updated and build passed 2025-05-13 14:49:37 +08:00
Thomas Ning
b49f7de81f Improve the general performance of the Preshuffled GEMM V3 & delete the unnecessary instances (#2166)
* make the work compiled

* Solved the example code, but still have the profiler error

* Finished the feature

* Clang format and update the CHANGELOG

* solve the preshuffle v1 & v2 problem

* Comment Addressed

* Comment Addressed
2025-05-12 09:52:58 -07:00
Your Name
58f848cc07 Merge branch 'wjx/align_v3_pipeline' into fp4_gu_moe 2025-05-12 16:21:39 +08:00
aska-0096
6c761bf9b8 tempsave; buggy at passed 4 e8m0 to scaled mfma 2025-05-10 09:57:49 +00:00
Bartłomiej Kocot
6fddb5708c Add grouped conv fwd bias relu instances (#2179)
* Add grouped conv fwd bias relu instances

* fixes

* fix
2025-05-09 22:52:34 +02:00
aska-0096
7bde4b8d34 Add pipeline v3. Have some runtime issue and register spill 2025-05-09 09:47:22 +00:00
Andriy Roshchenko
cb27e7c77f Ensure MX GEMM Instances can be Cross-Compiled for Multiple Architectures (#2171)
* Re-enable MX GEMM instances

* Fix compilation error when building MX GEMM for multiple architectures
2025-05-08 13:26:03 -06:00
aska-0096
b2efb06315 Spilt the fp4 target. Fix the known bugs. 128x128x128 sanity checked; remove prints 2025-05-08 15:07:33 +00:00
aska-0096
32e6c8aad3 16x16x128 correct; 64x64x128 failed 2025-05-08 09:42:53 +00:00
Ding, Yi
6519ba3aa9 wip3 2025-05-08 07:44:22 +00:00
Bartłomiej Kocot
397b9080a2 Move 16x16 grouped conv fwd instances from comp header (#2165)
* Move 16x16 grouped conv fwd instances from comp header

* Improvements
2025-05-07 08:04:31 -07:00
Ding, Yi
8d51a4ae96 wip2 2025-05-07 09:40:47 +00:00
Rostyslav Geyyer
8a0d659f92 Add FP4 MX MFMA tests (#2151)
* Add conversion tests

* Fix ctor

* Fix nan logic

* Fix conversion logic

* Permute packed f4_t values

* Fix conversion to float, repack vector elements

* Fix device tests

* Permute elements in a vector

* Add a repro test

* Add a conversion for a repro test

* Update test vectors

* Update conversion

* Fix the test

* Update test vector generator

* Fix vector sr conversion

* Permute conversion args

* Update conversion

* Test

* Fix packing

* Simplify conversion function

* Pack conversion in a loop

* Pack conversion in a loop

* Pack another conversion in a loop

* Pack one more conversion in a loop

* Pack the last conversion in a loop

* Clean up

* Add ops

* Add tests

* Add missing utils

* Update reference mx gemm

* Add f4x2 init mode

* Update host tensor utils

* Update chunk size for f4x2

* Add non scaled ops

* Add a type utility

* Update non scaled reference kernel

* Add non scaled tests

* Debug mfma arguments

* Add more debug info

* Update chunk size

* Update data layout

* Add more debugging

* Fix B stride

* Fix reference gemm

* Fix build

* One more reference fix

* Add more debug info

* Disable some tests

* Enable tests

* Add fp4 dimensions

* Update reference kernels

* Temp edits

* Remove leftovers

* Fix conflicts

* Clean up

* More clean up

* Revert "More clean up"

This reverts commit d8d35a0846.

* Add layouts to tests

---------

Co-authored-by: Andriy Roshchenko <107577548+andriy-ca@users.noreply.github.com>
2025-05-06 09:24:00 -05:00
Andriy Roshchenko
79beaacdd1 Restrict MX GEMM instantiation to GFX950 arch (#2157) 2025-05-05 08:18:22 -07:00