* Run ctest with --output-on-failure
* Fix synchronization issues in bwd pipelines
The bwd kernel reuses the same area of LDS for ds (SGrad), bias and
dbias (BiasGrad). This means that there must be block_sync_lds between
loading one tensor and storing another to the same area.
Heavy instructions like MFMA/WMMA and global loads are executed between
reuses of the same memory so in MOST cases loading is finished by all
warps before storing is started. However, sometimes warps progress at
different speeds.
Running the tests multiple times and, preferably, with multiple
processes on the same GPU helps to trigger this issue:
bin/test_ck_tile_fmha_bwd_bf16 --gtest_repeat=-1 --gtest_shuffle --gtest_throw_on_failure
* Selective test filter initial commit.
* Expanded folder paths for parsing ninja dependencies.
* Fixing default branch name in the test evaluation script.
* Fixing paths for robustness and adding ctest command to the launch script.
* change jenkins file and few tests to upgrade CI
* Setting ninja build path.
* Fixing typo in Jenkinsfile, and wrong paths.
* Fixing typo in launch script.
* add few more tests to check CI logic
* Fixing header for shell script.
* turn off performance test by default, add option to run all unit tests
* revert dummy changes in source code to trigger tests
* make sure develop branch runs all unit tests
---------
Co-authored-by: Vidyasagar Ananthan <vidyasagar.ananthan@amd.com>
