Commit Graph

128 Commits

Author SHA1 Message Date
Illia Silin
7d6a0220e0 turn on the ck_tile gemm tests by default (#1849)
[ROCm/composable_kernel commit: dcbfa79542]
2025-01-30 07:03:48 -08:00
Illia Silin
11eab9ca17 disable inductor codegen tests on legacy OS (#1816)
[ROCm/composable_kernel commit: 8c29e06f3c]
2025-01-15 12:11:54 -08:00
Max Podkorytov
971203e15a fix parsing instances for pt inductor (#1796)
add unit test for gen instances for gemms

add unit tests for conv and batched gemms

add unit test for preselected gemm instances

apply ruff lint

add license header for the unit test

add inductor pytest to CI

verbose pip install

switch the directory before installing python packages

move the inductor codegen test

try yet another workdir

Update Jenkinsfile

The directory looks right, fixing pip module not found by invoking pip directly

Update Jenkinsfile

invoke pytest directly since the module is not found

Update Dockerfile

Install setuptools

update package structure

bump setuptools

maybe fix data path for library sources

fix library search path for conv instances

fix path in pyproject definition

compare path used in gen_instances with one in pyproject.toml; fix the difference

Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>

[ROCm/composable_kernel commit: c0b90f130f]
2025-01-13 13:51:08 -08:00
Thomas Ning
f273d0a699 Ck tile/gemm perf measure (#1750)
* Finished adding the performance benchmark for ck tile gemm

* Fix the executable rename problem

* fix the executable name error

* delete the unsupported layout combinations

* Update run_full_test.sh

* Update benchmark_mem_pipeline.sh

* Update benchmark_basic.sh

* change the executable of gemm_universal

* change ck_tile_gemm script permissions

* Addressed the comment

* Addressed the comment

* Fixed the comments

* Fixed Comment

* roll back the malfunctioned change

* Fix the Typo

* finalize the tile_gemm_fp16 performance monitoring

* fix the stash names for ck_tile gemm logs

* change the stashing logic

* change stashing syntax

---------

Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
Co-authored-by: illsilin <Illia.Silin@amd.com>

[ROCm/composable_kernel commit: 73a076eee1]
2025-01-09 17:41:49 -08:00
Illia Silin
d9040038bc upgrade sqlalchemy version (#1748)
* upgrade sqlalchemy version

* replace the connection with engine in to_sql call

* change the hipTes=nsor ctest syntax

[ROCm/composable_kernel commit: fdfe210230]
2024-12-15 16:25:21 -08:00
Illia Silin
fe12a2dc1a Upgrade to Ubuntu22.04 as default OS. (#1738)
* upgrade to ubuntu 22.04

* try adding -u roof docker options for ubuntu 22

[ROCm/composable_kernel commit: 90d8410d56]
2024-12-10 08:48:51 -08:00
Illia Silin
b1475239ff build CI for gfx12 by default (#1734)
[ROCm/composable_kernel commit: 23cf2026b4]
2024-12-09 14:11:20 -08:00
Illia Silin
468603a2e6 update CI timeout limits (#1733)
[ROCm/composable_kernel commit: 2f088b8707]
2024-12-09 09:32:14 -08:00
Illia Silin
a6fe7d5f3f Refactor CI performance tests. (#1726)
* merge the build and performance tests CI stages together

* add gemm performance test on gfx11/gfx12

* add suffices to distinguish gemm performance logs from different archs

* use smaller gemm set in CI for gfx10/gfx11/gfx12

* disable performance tests on gfx1030

* fix the shashing logic

* fix finding python3 for mha instances

[ROCm/composable_kernel commit: 355893cdd8]
2024-12-06 13:04:25 -08:00
Illia Silin
f001318d1f Upgrade default compiler to ROCm6.3 (#1723)
* upgrade to rocm6.3 compiler

* Proposed solution to convnd test failures in ROCm 6.3

---------

Co-authored-by: Andriy Roshchenko <andriy.roshchenko@amd.com>

[ROCm/composable_kernel commit: 86990558e3]
2024-12-05 17:29:12 -08:00
Illia Silin
fd183719a4 Reduce docker size and build time in CI. (#1699)
* refactor docker build in CI

* add Dockerfile.compiler

* add input args to Dockerfile.compiler

* rearrange the docker args

[ROCm/composable_kernel commit: aa6e2087f5]
2024-11-28 10:42:19 -08:00
Illia Silin
31e3b0ad98 update mainline compiler branch name (#1696)
[ROCm/composable_kernel commit: cb8c7f42d6]
2024-11-26 14:58:35 -08:00
Illia Silin
014f1cefd5 add --squash flag when building dockers (#1686)
[ROCm/composable_kernel commit: 19d4b79039]
2024-11-22 17:16:08 -08:00
Illia Silin
804c8701e8 remove gfx940;gfx941 from default target lists (#1640)
[ROCm/composable_kernel commit: 54440cf562]
2024-11-05 13:56:20 -08:00
Paul Fultz II
c459c26388 Build codegen as standalone (#1556)
* Build codegen as standalone

* Add exception for device tests

* Use local filesystem header

* add a codegen test CI stage and daily build

---------

Co-authored-by: illsilin <Illia.Silin@amd.com>
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>

[ROCm/composable_kernel commit: 10158b0ffd]
2024-10-15 13:20:42 -07:00
Illia Silin
e125c93318 [CI] remove the --rm docker container flags (#1568)
[ROCm/composable_kernel commit: 11444e4cf2]
2024-10-11 14:29:46 -07:00
Illia Silin
985156e572 remove gfx12 targets from daily builds with rocm6.2 (#1560)
[ROCm/composable_kernel commit: cfac9497e2]
2024-10-09 10:18:05 -07:00
Illia Silin
881bc2c930 Fix build logic using GRU_ARCHS. (#1536)
* update build logic with GPU_ARCHS

* fix the GPU_ARCHS build for codegen

* unset GPU_TARGETS when GPU_ARCHS are set

[ROCm/composable_kernel commit: 7d8ea5f08b]
2024-10-07 08:18:23 -07:00
Illia Silin
7394671138 re-enable the FMHA performance monitoring (#1539)
[ROCm/composable_kernel commit: 11b7a4db00]
2024-10-01 13:17:55 -07:00
Illia Silin
299db8fc86 make CK CI use different git credentials (#1529)
[ROCm/composable_kernel commit: 65f8d1440f]
2024-09-25 09:05:48 -07:00
Illia Silin
f969220365 Add a daily CI build with legacy dockers. (#1525)
* add an option to build CK with legacy dockers

* change the custom docker settings

* add environment varianble for custom docker

* use a new variable for legacy docker name

* new way to pass docker names for legacy OS

* add legacy docker check in the Build_CK function

* change groovy syntax

* add a check for legacy docker in getDockerImage

* make sure the legacy docker name is not empty

* remove the dumb-init call

* disable the tests in legacy OS dockers

* disable tests in legacy dockers

* use a different way to disable tests in legacy dockers

* rearrange the CI stages for legacy OS

* use different way to disable tests in legacy dockers

* update LD_LIBRARY_PATH for legacy dockers and add cron job

* update LD_LIBRARY_PATH at docker launch

* change the sytax for setting LD_LIBRARY_PATH

[ROCm/composable_kernel commit: f16ebf82d4]
2024-09-23 09:03:55 -07:00
Illia Silin
8815f1f974 Add rocm6.3_rc1 docker image (#1518)
* add image for rocm6.3_rc1

* fix deb package url

[ROCm/composable_kernel commit: 1658c0dc11]
2024-09-17 15:59:26 -07:00
Thomas Ning
84f3413bb2 Ck tile GPU verification sample develop & Add the CK TILE GEMM to the CI/CD test (#1505)
* Finished the feature of gpu verification

* Add the ck_tile_gemm test in the CI CD

* add the include of tensor_layou in reference_gemm

* Comment Addressed

* split ck_tile fhma and gemm tests into separate stages

* restructure the reference gemm

* restructure a new reference_gemm api that could read the device mem

---------

Co-authored-by: carlushuang <carlus.huang@amd.com>
Co-authored-by: illsilin <Illia.Silin@amd.com>

[ROCm/composable_kernel commit: 844f5a1712]
2024-09-14 21:08:40 +08:00
Illia Silin
30073ffa43 make sure to rebuild compilers if they changed (#1504)
[ROCm/composable_kernel commit: e07f1108c0]
2024-09-12 07:49:55 -07:00
Illia Silin
c53d851c22 Enable daily ninja build traces. (#1487)
* add ninja trace to CI builds

* fix ninja trace logic

* update the ninja trace logic in jenkins file

* limit the number of threads to run ninja build

* use ninja for installation after build

* update the path to ninjatracing tool

* use ninja to run check when using build trace

* fix jenkins logic

* fix typos

* set proper setup_args for all stages

* fix ninja syntax

* replace ninja check with ninja test

* enable ninja tracing with mainline and staging compilers

[ROCm/composable_kernel commit: 19d22e60c1]
2024-08-26 13:27:01 -07:00
Bartłomiej Kocot
0f7c45915c Add performance and large tensor tests for grouped conv (#1456)
* Add performance and large tensor tests for grouped conv

* Resize tests

* Resize tests

* update the python script to parse the grouped_conv results

* Remove int8 tests

* change bwd wei layout

---------

Co-authored-by: illsilin <Illia.Silin@amd.com>

[ROCm/composable_kernel commit: 2581727d2a]
2024-08-16 07:48:30 -07:00
arai713
576bdcc359 Codegen build w/CK (#1428)
* initial push

* cleaned up compiler errors

* removed commented code

* build codegen folder only for gfx9 targets

* remove separate stage for codegen tests from CI

* removed commented code from CMake

---------

Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
Co-authored-by: illsilin <Illia.Silin@amd.com>

[ROCm/composable_kernel commit: da214a5a58]
2024-08-09 08:15:06 -07:00
Illia Silin
2b8bd9450a Enable CI on gfx12. (#1454)
* enable CI build and test on gfx1201

* skip DL kernels in CI for gfx12

* only run CI on gfx12 if rocm version >= 6.2

* remove the rocm version check for CI on gfx12

* add a switch for CI builds on gfx12

[ROCm/composable_kernel commit: 4a5ab67871]
2024-08-08 16:29:15 -07:00
Illia Silin
b8eddc3771 upgrade to rocm6.2 as new default compiler (#1448)
[ROCm/composable_kernel commit: 5df10432d8]
2024-08-07 09:38:43 -07:00
Illia Silin
b18480eab1 Run CK_TILE FMHA benchmarks and collect the performance data. (#1447)
* run ck_tile benchmarks after the smoke tests and store logs

* change the path of fmha benchmark logs

* change the way of stashig ck_tile fmha logs

* prevent the errors in stages where no logs are generated

* fix the ck_tile fmha log names and headers

* generate the fmha performance logs in the root folder

* change jenkins scrip arguments format

* use exact file names for stashing

* modify scripts to process FMHA performance results

* unstash FMHA logs before parsing them

[ROCm/composable_kernel commit: 12c1f68dd9]
2024-08-07 08:18:26 -07:00
Illia Silin
a5c15d2b51 [CI][Jenkins] delete CI docker container upon exit (#1437)
[ROCm/composable_kernel commit: f31ba04afc]
2024-08-05 08:13:56 -07:00
Illia Silin
4cfc157a28 Add compiler flags for ROCm versions 6.2+ (#1429)
* add compiler flags to fix compiler issues

* fix typo.

* disable test_smfmac_op on all devices except gfx942

* specify full path to compiler in CI

[ROCm/composable_kernel commit: d311c95396]
2024-08-01 08:27:52 -07:00
Illia Silin
4488950d01 Add CK_TILE tests to daily CI builds. (#1381)
* add ck_tile tests to CI

* build and run ck_tile tests on gfx90a and gfx942 in parallel

* fix groovy syntax

* turn ck_tile tests OFF by default

* skip creating the build folder

* build ck_tile examples with 64 threads

* build ck_tile examples with cmake-ck-dev.sh script

* add video group to docker on mi300

* do not retry to rebuild the early CI stages

* help prevent jenkins false failure

* restore cron trigger

[ROCm/composable_kernel commit: 98a01bbc72]
2024-07-11 13:22:40 -07:00
Illia Silin
3b49b68660 [Jenkins] restore cron jobs (#1380)
* test the cron trigger

* fix the cron jobs

* restore the list of cron jobs

[ROCm/composable_kernel commit: f914c228c6]
2024-07-11 10:28:11 -07:00
Jun Liu
fa73739812 Fix issue with multiple targets and remove smfmac tests from unsupported test targets (#1372)
[ROCm/composable_kernel commit: 959073842c]
2024-07-03 23:34:38 -07:00
Illia Silin
57ae3ae99f Merging the gfx12 code into public repo. (#1362)
[ROCm/composable_kernel commit: 941d1f7ce0]
2024-06-27 00:33:34 -07:00
Illia Silin
11ce5b2508 disable the hipTensor test by default, only run once daily (#1321)
[ROCm/composable_kernel commit: 3fa7e2a6c4]
2024-06-03 14:07:30 -07:00
Illia Silin
237c390a30 Build CK library for all supported targets. (#1312)
* test library build for all supported targets

* increase the number of threads to build lib in CI to 64

[ROCm/composable_kernel commit: 34f3dfdd61]
2024-05-28 12:36:06 -07:00
Illia Silin
254758813f Code clean-up (#1285)
* code clean-up

* remove the profiling output samples

[ROCm/composable_kernel commit: 566b6480a2]
2024-05-10 09:41:39 -07:00
Illia Silin
c02ecc0cd1 [CI] Focus CI stages on MI200 nodes for resource optimization (#1273)
[ROCm/composable_kernel commit: f0bf1e3125]
2024-05-01 10:07:14 -07:00
Illia Silin
8e9c95c520 [CI][Tests] Add a daily cron job to build CK instances for gfx9;gfx10;gfx11. (#1271)
* add a daily build for instances for gfx9;gfx10;gfx11

* fix jenkins logic for instances only build

* fix the path for instance_only build

* reduce the number of build threads to 32

[ROCm/composable_kernel commit: f6b3f4715d]
2024-04-30 14:44:30 -07:00
Illia Silin
433f7857d7 Make daily cron jobs use the rocm6.1 compiler. (#1253)
* add rocm6.1 docker and make it default for CI

* fix typo

* move the rocm6.1 image into public dockerhub repo

* upgrade daily cron jobs to use rocm6.1

[ROCm/composable_kernel commit: 930f889c34]
2024-04-18 09:40:21 -07:00
Illia Silin
8986d38014 Upgrade to ROCm6.1 and turn on the -enable-post-misched=0 compiler flag. (#1250)
* add rocm6.1 docker and make it default for CI

* fix typo

* move the rocm6.1 image into public dockerhub repo

[ROCm/composable_kernel commit: caae537d8e]
2024-04-18 11:10:23 -05:00
Haocong WANG
ceaecc86ca [GEMM] Gemm universal device operation (#1154)
* Optimize GEMM on MI200/300:
1. Add new blockwise gemm pipeline
2. Add irregular splitk intances

* clang format + typo fix

* Fix a bug

* initial commit

* Add more instances to irregular splitk

* blkgemm pipeline v1~4 prototype

* Sanity Checked. Known issue:
1. Poor performance of splitk
2. Register spill on blkgemmpipeline v3

* Sanity and Performance fix:
1. fix a bug related to sanity in grouped b2c mapping
2. fix a bug related to sanity and performance in splitk offset

* Sanity and API update:
1. Remove prefetch stage
2. Fix valid check bug
3, Add first gemm_universal instance into ckProfiler

* Add NN instances for gemm universal

* 1. Add NT instances for gemm_universal
2. Fix a bug about Kpadding in gemm_universal

* Fix a bug regarding padding Odd K number

* remove kernel print

* Fix KPadding bug...

* Update safety check

* another try to fix kpadding..

* Sanity checked

* new instances..

* clang format+typo fix

* remove clang format script's change

* Add non-hotloop compile option

* 1. Add fp16xfp8 example
2. pull packed convert f8 from pr1150

* Some miscs.. opt and fix

* Add pipeline description docs

* Split universal gemm instance library to cut profiler compiling time

* uncomment cmakefile

* Fix a bug caused by blockwise_gemm_pipe_v2

* reduce default splitk to 1

* Add 224x256x64 tile size

* update, including:
1. Experiment pipeline 5~7
2. Optimization for pipeline 4
3. Organized instance library

* temp save

* temp save

* Permuted lds layout, sanity and function checked

* clang format

* Move OOB check from RunRead to RunWrite, for better software pipeline.
TODO: agpr spill when NN layout

* clangformat

* A/B splitpipe scheduler for v3

* Fix two bugs

* bug fix

* fix a bug in oob check

* Example for mixed fp16_fp8 gemm

* Clean experimental code blocks

* Add mixed precision gemm into profiler

* tempsave

* optimize m/n major lds layout

* Add RRR GEMM  mixed precision instances

* Optimize f8 matrix transpose

* Add test_gemm_universal

* A/B spilt schedule for blkpip v5

* Take ds_read2 into iglp scheduling scheme

* format

* fixed cmake

* Add llvm-option into CI cmake flag

---------

Co-authored-by: Jing Zhang <jizhan@amd.com>

[ROCm/composable_kernel commit: f83e9701e9]
2024-04-13 21:03:18 -05:00
Illia Silin
cb9b04b34a allow the CI to pass even if can't connect to db (#1214)
[ROCm/composable_kernel commit: cc1f733d0e]
2024-03-22 15:39:11 -07:00
Illia Silin
cbc9a5c809 Fix a couple of docker issues. (#1206)
* do not install sccache by default, only install rocm-llvm-dev for rocm6.1

* add sccache flag to docker build options

[ROCm/composable_kernel commit: f52109531b]
2024-03-19 08:38:52 -07:00
Illia Silin
4844e407ee Re-enable the performance tracking in CI. (#1203)
* test CK with rocm6.1 RC2

* add docker credentials for pull

* update the performance db name

* use environment variable for db name

* add rocm-llvm-dev package to ck docker

* turn off verification for daily performance runs

* do not stash ckProfiler on MI300 node

* add processing of mixed gemms to qa, fix parsing of splitk gemm logs

* fix the splitk gemm log file name

* turn the timing on for splitk gemm performance

[ROCm/composable_kernel commit: bdcd037428]
2024-03-18 09:48:29 -07:00
Paul Fultz II
871810b885 Add host lib (#1134)
* Format

* Format

* Format

* Remove const

* Use the right template

* Format

* Format

* add row/col instances

* Add missing file

* fixed

* Format

* Updates

* Format

* fixed rrr layout

* Format

* Update test and embed modules

* Restore older version

* Update year

* Set -fPIC

* Format

* Use double for isnan

* rename host folder to codegen + minor fix

* add codegen CI test

* add option to build components without building CK

* fix the groovy syntax

* fix typo

* use the correct function for the codegen stage

---------

Co-authored-by: Jing Zhang <jizha@amd.com>
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
Co-authored-by: illsilin <Illia.Silin@amd.com>

[ROCm/composable_kernel commit: 8eff4d62b6]
2024-03-05 17:08:43 -08:00
Illia Silin
7241d5ad6a [CI] Add CI build and test stage on MI300. (#1185)
[ROCm/composable_kernel commit: cf86621170]
2024-03-05 10:42:16 -08:00
Illia Silin
baa25bcc4b add docker credentials before pushing image (#1165)
[ROCm/composable_kernel commit: a78be3f69e]
2024-02-12 16:11:32 -08:00