* updating codegen build for MIOpen access: adding .cmake for codegen component
* updating CMake
* adding in header guards for some headers due to issues with hiprtc compilation in MIOpen
* some more header guards
* putting env file in header guard
* cleaning up some includes
* updated types file for hiprtc purposes
* fixed types file: bit-wise/memcpy issue
* updating multiple utility files to deal with standard header inclusion for hiprtc
* added some more header guards in the utility files, replacing some standard header functionality
* added some more header guards
* fixing some conflicts in utility files, another round of header guards
* fixing errors in data type file
* resolved conflict errors in a few utility files
* added header guards/replicated functionality in device files
* resolved issues with standard headers in device files: device_base and device_grouped_conv_fwd_multiple_abd
* resolved issues with standard headers in device files: device_base.hpp, device_grouped_conv_fwd_multiple_abd.hpp, device_grouped_conv_fwd_multiple_abd_xdl_cshuffle.hpp
* added header guards for gridwise gemm files: gridwise_gemm_multiple_abd_xdl_cshuffle.hpp and gridwise_gemm_multiple_d_xdl_cshuffle.hpp
* fixed issue with numerics header, removed from transform_conv_fwd_to_gemm and added to device_column_to_image_impl, device_grouped_conv_fwd_multiple_abd_xdl_cshuffle, device_grouped_conv_fwd_multiple_abd_xdl_cshuffle_v3, device_image_to_column_impl
* replaced standard header usage and added header guards in block to ctile map and gridwise_gemm_pipeline_selector
* resolved errors in device_gemm_xdl_splitk_c_shuffle files in regards to replacement of standard headers in previous commit
* added replicated functionality for standard header methods in utility files
* replaced standard header functionality in threadwise tensor slice transfer files and added header guards in element_wise_operation.hpp
* temp fix for namespace error in MIOpen
* remove standard header usage in codegen device op
* removed standard header usage in elementwise files, resolved namespace errors
* formatting fix
* changed codegen argument to ON for testing
* temporarily removing codegen compiler flag for testing purposes
* added codegen flag again, set default to ON
* set codegen flag default back to OFF
* replaced enable_if_t standard header usage in data_type.hpp
* added some debug prints to pinpoint issues in MIOpen
* added print outs to debug in MIOpen
* removed debug print outs from device op
* resolved stdexcept include error
* formatting fix
* adding includes to new fp8 file to resolve ck::enable_if_t errors
* made changes to amd_wave_read_first_lane
* updated functionality in type utility file
* fixed end of file issue
* resovled errors in type utility file, added functionality to array utility file
* fixed standard header usage replication in data_type file, resolves error with failing examples on navi3x
* formatting fix
* replaced standard header usage in amd_ck_fp8 file
* added include to random_gen file
* removed and replicated standard header usage from data_type and type_convert files for fp8 changes
* replicated standard unsigned integer types in random_gen
* resolved comments from review: put calls to reinterpret_cast for size_t in header guards
* updated/added copyright headers
* removed duplicate header
* fixed typo in header guard
* updated copyright headers
---------
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
[ROCm/composable_kernel commit: 2e3183af4f]
* Add spatially local tile partitioner
* Use 1D Grid size & create partitioner object.
* Docs & use 1D partitioner in example.
* Clang format.
* Change kernel grid size
Now: X is the # of output C-tiles,
Y is the batch count
Z is the splitK
* Formatting & more doc.
* Clang format.
* Fix batched gemm test. Use 1d partitioner.
* Move condition.
* FIx ctor.
* clang-format.
[ROCm/composable_kernel commit: ce448002ee]
* Change flag from CK_WORKAROUND_DENORM_FIX to CK_GFX90A_DENORM_WORKAROUND for more clarity. Also changed the definition macros to be more clear.
[ROCm/composable_kernel commit: d6a4605e1c]
* Refactor universal gemm policy.
* Adapt example to refactor changes.
* Introduce static encoding pattern
* Adding shuffled encoding patterns.
* Fix err in reverse tuple.
* Add transpose_tile2d
* Small refactoring + doc
* Enable reading on contiguous dimension in all layouts.
* Transpose A/B register tile if needed for comp v3 pipeline.
* Take contiguous dim size when calculating dram vector load size.
* A/B smem pack size taken from WarpGemm attributes
* Update B LDS layout and setup tile distribution pattern at class level.
* Fix static assert.
* Fix errors in examples.
* Formatting & fix IsTranspose
* Fix VectorSize & refactor.
* Add error loging messages.
* Fix VecLoadSize and TranspseC for mem pipeline.
* Update unit-tests & disable mem pipeline.
* Clang format
* Update include/ck_tile/core/tensor/tile_window.hpp
Co-authored-by: jakpiase <jakub.piasecki@amd.com>
* Fix compilation and reviewers comments.
* Refactor unit-test. Fallback to non-universal gemm.
Need to use GemmPipelineAGmemBGmemCRegV1 for now,
since GemmKernel is now supporting also non-K major vector reads.
---------
Co-authored-by: jakpiase <jakub.piasecki@amd.com>
[ROCm/composable_kernel commit: 39dc25a9b8]
* not using structures under ck_tile/ops for ck_tile/host
* update as constexpr function
* Rename fn
* Update other examples.
---------
Co-authored-by: Adam Osewski <19374865+aosewski@users.noreply.github.com>
Co-authored-by: Adam Osewski <Adam.Osewski@amd.com>
[ROCm/composable_kernel commit: 5b9b083dbc]
* smoke and regression targets working with tests
* test filters work for both examples and test
* removed uneccesary comments
* added a missing comment
* added a missing comment
* fixed typo in the comments
* updated README
* Update PULL_REQUEST_TEMPLATE.md
updating the template for future addition of test cases
* Update PULL_REQUEST_TEMPLATE.md
[ROCm/composable_kernel commit: 54de3e55e1]
* Add shortcut to RMSNorm
* Modify test for adding shortcut for RMSNorm
* Add fused parameter into tests
* 1. Add YDataType. 2. rmsnorm2d_fwd_traits_ from rmsnorm2d_fwd.hpp to rmsnorm2d_fwd_api.cpp and rmsnorm2d_fwd_instance_common.hpp
* 1. Supports various stride and percisions.
* Add support of Epilogue
* Add fuse and epilogue support to rmsnorm ref
* Modify rmsnorm example
* Refactor tests/examples
* Bug fix for newly added tests/examples
* Bug fix for new tests 2
* Modify smoke test scripts
remove dbg code
* Supports non-smooth dyanmic quant
* Update Rmsnorm2dFwd::GetName()
* rename xscale and prec_sx to smoothscale and prec_sm
Bug fix after rename
Remove files
* change example_rmsnorm2d_fwd.cpp
* update performance calculator
* Fix issue in two-pass when fuse add is enabled
* Remove comment of beta
---------
Co-authored-by: rocking <ChunYu.Lai@amd.com>
[ROCm/composable_kernel commit: 04dd314883]
add unit test for gen instances for gemms
add unit tests for conv and batched gemms
add unit test for preselected gemm instances
apply ruff lint
add license header for the unit test
add inductor pytest to CI
verbose pip install
switch the directory before installing python packages
move the inductor codegen test
try yet another workdir
Update Jenkinsfile
The directory looks right, fixing pip module not found by invoking pip directly
Update Jenkinsfile
invoke pytest directly since the module is not found
Update Dockerfile
Install setuptools
update package structure
bump setuptools
maybe fix data path for library sources
fix library search path for conv instances
fix path in pyproject definition
compare path used in gen_instances with one in pyproject.toml; fix the difference
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
[ROCm/composable_kernel commit: c0b90f130f]
* refactor the block_gemm_areg_breg_creg_v1 and add the v2 policy with 2x2 warp gemm
* Finished the 2x2 warp gemm policy and the block selection mechanism
* Clang format
* address poyen's comment
* Address feedbacks
* Fixed the compilation issue
* Change the function name
[ROCm/composable_kernel commit: 5d671a5fc4]
* Observed a 2x perf improvement with kBlockSize = 256
* Using 512 threads may lead to redundant computations
[ROCm/composable_kernel commit: 0b8f117f1a]
* Update for fmha_fwd qs_ks_vs pipeline
* Remove _builtin_amdgcn_sched_barrier(0)
* Move p_compute to p converting earlier for trying to increase vgprs re-using
* Enable GetQKBlockGemm to use WarpGemm-16x16x16 for QLoadOnce==false situation
* Re-add __builtin_amdgcn_sched_barrier(0)
---------
Co-authored-by: Po Yen Chen <PoYen.Chen@amd.com>
[ROCm/composable_kernel commit: 3d50f57f43]
* Disable building DPP kernels by default
* Disable building dpp instances, examples, or tests if DPP_KERNELS is not set
* Add new DPP_KERNELS flag to readme
[ROCm/composable_kernel commit: 26b3829c02]
