Ville Pietilä
70d57ca8b9
Remove separate packed cast step.
2025-09-03 10:57:16 +00:00
Ville Pietilä
d56d7bc821
Use fused packed cats.
2025-09-02 12:00:21 +00:00
Ville Pietilä
d22ec6633a
Fused packed cast improvements.
2025-09-01 14:57:42 +00:00
Ville Pietilä
c69539fe3c
Optimize LDS write order for packed cast.
2025-08-28 15:01:55 +00:00
Ville Pietilä
9f66d9fbca
Packed cast improvement.
2025-08-27 14:32:00 +00:00
Ville Pietilä
b82e68c45f
Bug fix.
2025-08-27 14:30:13 +00:00
Ville Pietilä
f1a7cfba26
Packed cast imrovement.
2025-08-27 14:23:05 +00:00
Ville Pietilä
90b90dff08
Vectorized packed cast.
2025-08-27 13:29:38 +00:00
Ville Pietilä
54302c6f77
Remove obsolete version of the packed cast.
2025-08-27 11:28:55 +00:00
Ville Pietilä
481df169f2
Add packed cast to gridwise gemm multi d.
2025-08-27 11:27:03 +00:00
Ville Pietilä
6092643e9b
Performat packed cast implementation.
2025-08-27 10:10:18 +00:00
Ville Pietilä
cfbd669455
WIP: Vectorized access.
2025-08-26 12:43:38 +00:00
Ville Pietilä
2302ea9bc6
Add the vectorized option for packed cast.
2025-08-26 12:31:01 +00:00
Ville Pietilä
905cfb6623
Use thread scratch buffer in bf16 conversion.
2025-08-26 09:33:50 +00:00
Ville Pietilä
1b858936cb
WIP: Packed cast using threadwise sratch.
2025-08-25 15:31:50 +00:00
Ville Pietilä
4e7f9f7908
Fix packed cast tensor slice transfer.
2025-08-22 10:26:59 +00:00
Ville Pietilä
de93a48b04
Add back the separate packed cast step.
2025-08-20 11:31:07 +00:00
Ville Pietilä
6fbe1895f1
Code clean-up.
2025-08-20 08:19:13 +00:00
Ville Pietilä
0d34572f20
Code clean-up.
2025-08-19 18:37:01 +00:00
Ville Pietilä
b48ae7e447
Add perf test. Fix packed bf16 cast implementation.
2025-08-18 11:37:14 +00:00
Ville Pietilä
6b2b5e7c7c
Small optimization to the packed cast.
2025-08-18 06:39:26 +00:00
Ville Pietilä
c0b8f66674
Add packed cast pipeline into gridwise gemm xdlops bwd weight.
2025-08-18 06:10:14 +00:00
Ville Pietilä
00a3ce734a
Integrate new packed cast threadwise tensor slice transfer into gridwise gemm pipelines.
2025-08-15 12:06:44 +00:00
Ville Pietilä
51af3d7bac
Fix a bug in the packed cast threadwise transfer.
2025-08-15 10:41:06 +00:00
Ville Pietilä
62c66a7d9c
WIP: packed bf16 cast v3.
2025-08-14 12:39:18 +00:00
Ville Pietilä
938ff298b4
Add more unit tests.
2025-08-14 11:33:02 +00:00
Ville Pietilä
3ecc8aae74
Add unit test for vectorized packed cast.
2025-08-14 08:41:35 +00:00
Ville Pietilä
ade741dd45
WIP: PackedCast v3.
2025-08-13 15:13:35 +00:00
Ville Pietilä
50e318e072
Fix logging.
2025-08-12 15:53:00 +00:00
Ville Pietilä
ae4c727bc5
Add packed bf16 cast for universal GEMM.
2025-08-12 15:52:49 +00:00
Ville Pietilä
cee7644c85
Working version 2 of the packed cast.
2025-08-12 12:46:01 +00:00
Ville Pietilä
6148d1c75f
WIP: Packed cast v2.
2025-08-11 15:18:30 +00:00
Ville Pietilä
c675563468
Addlogging and specific unit tests for bf16 and gfx950.
2025-08-08 08:59:12 +00:00
Ville Pietilä
c47b80580d
Fix build issues when __gfx950__ macro is enabled.
2025-08-08 08:01:42 +00:00
Ville Pietilä
4b8a559da9
Fixed packed_cast implementation for slice access.
2025-08-06 11:04:29 +00:00
Ville Pietilä
44202b9d32
WIP: Integration of packed cast into gridwise_gemm_xdl_cshuffle_conv_v3.
2025-08-05 15:12:36 +00:00
Ville Pietilä
e92c0bf68e
Initial integaration of packed cast.
2025-08-04 15:34:35 +00:00
Ville Pietilä
e06548675f
Fix a bug in packed cast asm. Add more unit tests.
2025-08-04 09:16:12 +00:00
Ville Pietilä
9769fa68a7
Packed BF16 cast with asm volatile.
2025-08-01 13:22:41 +00:00
Ville Pietilä
b2b991d431
Added conversion of two floats into a packed bf16 value.
2025-07-31 15:13:37 +00:00
Ville Pietilä
e962a41638
Automatic deduction of split-K value for grouped convolution ( #2491 )
...
* Split-K autodeduction for DeviceGroupedConvBwdWeight_Xdl_CShuffle and DeviceGroupedConvBwdWeight_Xdl_CShuffleV3.
* Split-K autodeduction for DeviceGroupedConvBwdWeightTwoStage_Xdl_CShuffle.
* Use simple best occupancy model to calculate the split-K.
* Handle split-K autodeduction in explicit gemm conv.
* Add unit tests for split-K autodeduction.
* Remove oversubscription.
* Small fixes.
* Added split-K autodeduction for DeviceGroupedConvBwdWeightMultipleD_Xdl_CShuffle.
* Run clang formatting.
* Fix error handling in the conv profiler.
* Add missing documentation for the autodeducted split-K values.
* Add split-K autodeduction to DeviceGroupedConvBwdWeight_Explicit_Xdl solver.
* Fix clang formatting and split-K profiler documentation.
* Rename max_occupancy value variable.
* Calculate grid size for split-K autodeduction directly from input array shapes and template params.
---------
Co-authored-by: Ville Pietilä <>
2025-07-31 12:08:45 +02:00
Bartłomiej Kocot
5b244105d9
Enable multiple D for grouped conv fwd large tensors ( #2572 )
2025-07-28 22:39:07 +02:00
linqunAMD
0782ee8eb3
Remove !defined(__HIP_DEVICE_COMPILE__) in CK kernel ( #2564 )
...
* Remove HIP_COMPILE_DEVICE
* add missing files
* fix clang format
---------
Co-authored-by: Lin, Qun <Quentin.Lin+amdeng@amd.com >
2025-07-28 13:01:07 -07:00
Illia Silin
504b101da3
upgrade from clang-format-12 to clang-format-18 ( #2568 )
...
* upgrade to clang-format-18
* update to clang-format-18 in pre-commit-config
2025-07-28 11:34:07 -07:00
Bartłomiej Kocot
685771b875
Enable bf16 RNE on gfx950 ( #2542 )
...
* Enable bf16 RNE for gfx950
* test bhalf
* fix
* fix
* Comments fixes
* fixes
* clean
* fix
2025-07-28 00:47:17 +02:00
Adam Osewski
c8eb2f995c
Add v3 support for Groupd fwd conv+bias+clamp & ckProfiler ( #2463 )
...
* Add logging to IsSupported.
* Less casting in AddClamp
* Conv+bias+clamp instances & profiler BF16
* Fix 3D instances & run just 1x for verification.
* :Run just once for verification conv fwd.
* ckProfiler conv fwd clampwq
* Remove exec bit & formatting
* Add support for MultiD for grouped conv fwd v3.
* Enable 2Lds.
* clean
* align instances
* align instances
* profiler fixes
* Fixes
* fix
* fix
---------
Co-authored-by: Adam Osewski <root@quanta-ccs-aus-f01-19.cs-aus.dcgpu >
Co-authored-by: Bartłomiej Kocot <barkocot@amd.com >
2025-07-25 10:34:31 +02:00
Enrico Degregori
b01a27ff22
Support b_scale: ( #2350 )
...
- extend pipeline v1 and v3
- add instances
- add tests
- add example
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com >
2025-07-24 18:49:58 -07:00
Illia Silin
9c04a55626
remove repetitive code ( #2562 )
2025-07-24 14:52:46 -07:00
Andriy Roshchenko
3421272f90
MX GEMM - FP6 Support in GEMM MX v3 Pipeline ( #2481 )
...
* Add GEMM MX BF6 example
* Fix BF6 type_convert
* Add type_convert for bf16x6
* Add compare operator to f4x2_pk_t
* Update README for 67_gemm_microscaling
* Fix host tensor initialization with integer values for FP8
2025-07-24 14:36:53 -04:00
Rostyslav Geyyer
c9886109b4
Update packed fp4 layout ( #2523 )
2025-07-21 16:58:59 -05:00
