* allow building CK for specific data types
* add CI build and test stage on Naiv3x without some int8 instances
* add missing gemm fp16 instances
* add the changes to the missed cmake file
* add empty lines at end of source files
* Do not build quantization client example on navi3 in CI
* disable batched_gemm_multi_d_int8 instances with DTYPES
* disable device_conv2d_bwd_data_instance with DTYPES
* fix ckprofiler for conv_bwd_data for int8
* properly isolate the conv_bwd_data int8 instances
* remove empty line
* enable gfx941/942 targets
* fix clang format
* fix the cmake logic for multiple targets
* fix cmake syntax for looping over targets
* add gfx941/942 support for gemm_xdl instances
* enable dl kernels on navi3
* do not build xdl tests and examples on Navi
* run tests before building everything on jenkins
* disable gemm_bilinear on gfx1030
* add gpu targets to installer on Navi
* put tests in the same order as before
* reduce the number of navi targets in CI
* build CI installed for gfx940 as well
* only build for MI300 during QA runs
* enable use of rocm5.5 release candidate 4
* upgrade to ROCM5.5 RC5
* try fix the PUB_KEY error, remove the cmake-data package
* upgrade to latest cmake version
* use private dockerhub repo for rocm5.5 rc5
* add missing bracket
* add new parallel stage on navi node
* dont run performance tests on navi, get rid of 9110 compiler
* only run navi build when not doing QA
* fix syntax
* use navi21 label
* dont stash profiler on navi nodes, scp deb package to ginger
* disable tests on navi nodes
* test posting a binary to ginger
* add sshpass and use it to copy deb package
* fix the scp example
* fix syntax
* debug the scp issues
* add jenkins user to docker
* dont try whoami
* change jenkins uid and add user with uid=1002
* try scp from the last stage on micimaster
* rename and stash the package, scp from micimaster
* test the QA cron parameter for compiler commit
* create separate dockers for latest and fixed amd-stg-open compiler versions
* change groovy syntax
* apply cron timers back to develop branch
* enable ccache and decouple it from MIOpen ccache use
* fix the ccache check script
* use another method to get server name
* fix syntax
* add quotes around the server name variable
* use check_host as function
* change syntax
* fix syntax
* test if server name is parsed correctly
* try different syntax
* check the env var value
* test new check node function
* add ROCMVERSION parameter and fix script syntax
* fix script syntax
* add missing instances of rocm version
* install ccache in the docker image
* do not check GPU in clang format stage, clean up old code
* update defaults and clean up
* add an option to select specific compiler commit
* change the logic of forcing building a docker
* add check for compiler commit in dockerfile
* compiler check syntax fix
* change compiler selection logic
* fix the new compiler build issue
* set new compiler as default, update dev-requirements
* fix jenkins syntax
* fix docker syntax
* get rid of hipcc.pl editing in jenkinsfile
* fix the hipcc.pl in both places
* try to fix the 10738 compiler linking bug
* fix syntax
* use dockerhub to store images
* use newer amd-stg-open commit as default
* build CK only once, use deb package in all subsequent stages
* update jenkins file
* change prefix for build_CK stage
* update writing deb metadata to control file
* update ubuntu source for docker, script syntax for deb package metadata
* try different way to create deb metadata
* clean up DEBIAN before creating one
* fix the CI folder names, fix splitK qa
* use correct docker in all stages, separate tests for splitK verification and performance
* clean old comments, change dir before packaging
* use different package syntax
* change packaging syntax
* package with cmake
* remove unnecessary build prefix
* get rid of unnecessary paths
* change paths during unpacking
* change script syntax while unpacking
* get rid of unneccesary steps
* get rid of comments in the scripts
* use double quotes for scripts
* add ccache during build, try dpkg -x
* pull and install each package separately
* use full package names
* try to use stashing for packages
* change stash/unstash syntax
* move unstash out of shell, run tests on any gpu node
* unpack each package separately
* try re-using existing workspace
* merge the build and test stages, only stash ckProfiler
* merge the build and test stages, only stash zipped ckProfiler
* fix syntax
* add GPU check before build and test, rename docker to usual name
* upgrade the OS and ROCM versions in CK docker
* add cxx flags to link code with rocm5.2 and ck-9110 compiler
* rename the docker image
* run ONNX gemms using init=1
* add processing for the onng_gemm and splitK_gemm
* add profile_onnx_gemm.sh
* add stderr to logfiles, add splitK and onnx gemm parsing
* enable splitK gemm wresults posting to db
* replace hipcc compiler with clang++
* build client app with hipcc
* build client app with clang
* add an option to build with hipcc ro clang
* fix the environment for client app
* fix setting up compiler in cmake_build
* change the way the compiler is set
* build docker in separate stage
* build docker with only one prefix
* add parallel statement
* add docker repo url
* fix the name of perf_conv_bwd_data log file
* allow selecting compiler version
* fix typo
* add Wno-deprecated flag for google tests
* change git repo, fix qa log files names
* change the git clone syntax
* use Omkar's git credentials
* try to use jenkins as git user
* try using illsilin username for gerrit repo with ssh key
* try new gerrit authorization
* change ssh key syntax
* try another way of passing ssh key to docker
* add mount ssh in dockerfile
* create .ssh folder
* move ssh-keyscan to later
* get rid of npm call
* build first docker image on master
* check the contents of the .ssh folder
* try replacing omkars creds with gerrit creds
* use open repo, clean up changes
* get rid of ssh default argument
* turn on full qa only on gfx90a, use int initialization
* change script syntax
* update script parsing clinfo, throw exception if 0 devices
* fix syntax
* try using toBoolean for the QA conditions
* run regular CI on MI100 only, use MI200 only for daily QA
* evaluate when conditions before agent
* launch QA on develop branch and update profile_reduce script
* update test script
* update script
* remove false dependency from dockerfile
* try removing rbuild completely
Co-authored-by: Chao Liu <chao.liu2@amd.com>
Co-authored-by: Chao Liu <lc.roy86@gmail.com>
* add verify flag and update scripts
* replace old check_error function with the new check_err
* fix syntax
* remove blank spaces
* remove empty line
* add check_err for tensors
* fix syntax
* replace tensors with vectors in check_err calls
* fix syntax
* remove blank spaces
* fix syntax
* add new line at end of file
* disable conv2d_bwd_weight test, add gpu check
* set check_gpu using export
* check GPU using runShell
* add definition of runShell
* fix script syntax
* reduce the number of threads, add full qa option
* run processing scripts in bash
* fix the branch and host names in performance scripts, add chronos
* replace parameterizedCron with cron
* archive the perf log files
* try to fix git call
* pass branch and host names as arguments into scripts
* fix script arguments
* fix script arguments
* process results on master
* fix pipeline
* add definition of gpu_arch
* run processing scripts in docker
* fix the brackets
* add agent master for the processing stage
* get rid of show_node_info call on master
* try using mici label instead of master, disable MI100 tests for now
* fix syntax
* simplify container for results processing
* remove node(master) from the process_results stage
* put all stages in original order
* change the agent label from master to mici for gfx908
* adding scripts for full perf test suite
* uncomment the sql queries
* fix typo and chmod a+x for scripts
* dos2unix for all new scripts
* disable verification in full performance test
* fix reduction scripts, add gfrouped_gemm hotfix
* fix the grouped_gemm hotfix and only run reduction for fp16
* change compiler flag syntax
* fix syntax
* add predefinition of dockerArgs
* avoid redefinitions of dockerArgs
* add blank space at the end of dockerArgs
* try to build with release compiler
* adding spaces inside if condition
* limit the number of threads for building 9110 compiler
* change the way HIP_CLANG_PATH is set
* remove the export command
* change the conditional ENV syntax
* set HIP_CLANG_PATH at docker run time
* update scripts for full qa
* enable the sql write query
* fix typo
* remove a comment from a script
* use pre-built docker instead of building a new one
* try docker.image.pull
* change syntax in docker.image()
* add 30 min timeout
* increase timeout to 3 hours
* move performance tests to first stage for testing
* set image variable to the new container name
* update image name
* check available images
* check available images in both places
* try different image name
* use image ID to refer to image
* run performance on gfx90a
* fix the gpu_arch labeling, add parameter
* move env vars out of stages
* add stand-alone performance script, MI200 tests, CU numbers
* dos2unix for run_perf_tests.sh
* try the new git credentials
* use env var for git credentials
* don't look up /sys/module/amdgpu/version
Co-authored-by: Chao Liu <chao.liu2@amd.com>
* use pre-built docker instead of building a new one
* try docker.image.pull
* change syntax in docker.image()
* add 30 min timeout
* increase timeout to 3 hours
* move performance tests to first stage for testing
* set image variable to the new container name
* update image name
* check available images
* check available images in both places
* try different image name
* use image ID to refer to image
* run performance on gfx90a
* fix the gpu_arch labeling, add parameter
* move env vars out of stages
* add stand-alone performance script, MI200 tests, CU numbers
* dos2unix for run_perf_tests.sh
* try the new git credentials
* use env var for git credentials
* use pre-built docker instead of building a new one
* try docker.image.pull
* change syntax in docker.image()
* add 30 min timeout
* increase timeout to 3 hours
* move performance tests to first stage for testing
* set image variable to the new container name
* update image name
* check available images
* check available images in both places
* try different image name
* use image ID to refer to image
* run performance on gfx90a
* fix the gpu_arch labeling, add parameter
* move env vars out of stages
* add stand-alone performance script, MI200 tests, CU numbers
* add resnet50 test to performance tests
* add blanks before gpu_arch in log files
* add resnet50 test with N=4 and process its results
* add ROCM and HIP versions to test tables
* uncomment the sql queries
* fix script syntax in jenkinsfile
* modify ckProfiler_gemm output
* fix syntax
* change ckProfiler output and return 0
* fix syntax
* output datatype
* fix syntax
* output datatype in another way
* fix syntax
* fix syntax
* test return values of ckProfiler
* add layout info and tests, make sure ckprofiler returns 0
* fix syntax
* change layout output
* fix syntax
* fix syntax again
* update script to process perf results
* rearrange jenkins stages
* fix typo
* add python packages to Docker file
* adding setuptools-rust package
* modify parsing for new test parameters
* test db credentials on jenkins
* fix syntax
* update python script to handle incomplete lines
* ungrade python to 3.8 and write the gemm_params table
* add sqlalchemy package to docker
* move perf data processing to master node
* move the master node inside a steps region
* add new stage for result processing
* move results processing to separate stage
* reduce number of tests to speedup debugging
* pass config to processPerfResults stage
* run script on master in a docker container
* replace show_node_info
* try loading docker on master node again
* use ansible node instead of master
* get rid of pymysql package
* try ssh connection using paramiko
* put back pymysql
* put the perf data processing back on the gpu node
* put back artifact definition
* archive the perf_log before parsing
* clean up jenkinsfile, fix parsing
* fix typo
* enable all perf tests
* put all stages in original order, finalize script
* fix gpu_arch version
* update parsing script
* remove obsolete file causing merge conflict
* add performance test to jenkins pipeline
* fix typo
* fix the syntax in conv_fwd_util.cpp
* fix the error message syntax spacing
* fix the error message syntax spacing again
* run profile_gemm and archive results
* fix typo
* try to figure out the paths
* try to figure out the paths one more time
* skip the copying step
* build ckProfiler release only once
* change directory using dir
* fix dir syntax
* change the gemm parameters
* do not pipe script output to file
* try running ckProfiler directly
* fix typo
* use set +e
* run profile_gemm.sh || true
* run multiple gemms and parse results
* fix typo in jenkinsfile
* fix syntax
* add new gemm sizes, update scripts
* put all jenkins steps in original order
Co-authored-by: Chao Liu <chao.liu2@amd.com>
Co-authored-by: Chao Liu <lc.roy86@gmail.com>
