Aviral Goel
|
d85f065b15
|
chore(copyright): update copyright header for example directory (#3273)
* chore(copyright): update copyright header for codegen directory
* chore(copyright): update copyright header for example directory
|
2025-11-24 18:02:41 -08:00 |
|
Johannes Graner
|
d40b50b9d5
|
Update pre-commit to fixed versions, run remod for ck_tile (#2895)
* Fix ruff linter errors
* Fix remod dos2unix command
* Clang format
* Ignore utility in remod
* Run remod
* Specify clang-format version in pre-commit
* Specify ruff version
* Include PoolKernelArgs in reference_pool
* Add calculate_total_elements to reference batched contraction
* Fix calculate_total_elements declaration
* Refactor remod pre-commit hook
* Fix Aquant tests
---------
Co-authored-by: Illia Silin <98187287+illsilin@users.noreply.github.com>
|
2025-10-16 15:29:17 -07:00 |
|
Po Yen Chen
|
0cb2e06ddc
|
[CK_TILE] fmha forward split-kv + combine kernels (#1338)
* FA fwd dropout
* FA bwd
* epilogue reuse
* CMakeLists update
* [CK_TILE] support alibi (#1269)
* add alibi support
* fix code
* update code based on comment
* Support more hdim
* fix fp8 bias
* support seqlen_k=0 case
* remove unused printf
* fix format
---------
Co-authored-by: rocking <ChunYu.Lai@amd.com>
* now fwd/bwd can build
* bwd alibi
* add bwd validation stream_config
* update generated filenames
* update bwd kernel launch
* CK_TILE_HOST_DEVICE in philox
* Transpose -> transpose
* format
* format
* format
* Generate the instance for FA required
* format
* fix error in WarpGemm
* Add num_splits option and dummy split-kv api method
* Generate fmha_fwd_splitkv()
* Add SplitKV kernel codegen logics
* Add SplitKV combine kernel codegen logics
* Fix mismatched return type
* Clean-up code
* Replace sentinel value before storing
* Fix wrong layout of LSE/LSEacc/Oacc
* Format codes
* Fix o_acc memory error
* Fix wrong kBlockSize used in policy
* Reduce # of combine kernels
* Fix split-kv combine kernel name
* Fix wrong LDS indexing logics
* Fix wrong loop counter step logic
* Undo vector size changes
* Remove no-longer used field
* Remove in-consistent comment
* Remove debug statements in example
* Remove more debug statements
* Add constness to local variables
* Clearn up generate.py
* Fix unstable clang-format comment
* Remove unused include directive
* Use shorter template parameter name
* Enable non-split-kv blobs
* Update license date
* Print num_splits conditionally
* Undo disabling data types
* Remove unnessary tile size for fp8
* Fix wrong pipeline args for fp8
* Fix example output format
* Remove more debug code in combine pipeline
* Add stride kernel arguments for LSE/O acc workspace
* Re-order split-kv pipeline call operator arguments
* Pass LSE/O strides in kernel argument
* Re-order pipeline call operator arguments
* Use tensor_descriptor to locate LSEacc elements
* Support providing invalid element for tensor view
* Set invalid element value for LSEacc tensor view
* Remove hand-written store_tile() code
* Remove necessary value-overwrite logic
* Add transposed lds descriptor
* Support load_tile() for tile_window_with_static_lengths<>
* Undo removing necessary value-overwrite logic
* Use read descriptor to locate lds elements
* Simplify pipeline source code
* Add constraint to kMaxSplits
* Default use kMaxSplits=64 in generate.py
* Revert "Add constraint to kMaxSplits"
This reverts commit 0a2132d758.
* Revert "Default use kMaxSplits=64 in generate.py"
This reverts commit c7d9c80b77.
* Decide alignment by the padding parameter
* Remove no-longer used utility functions
* Remove not-working code
* Add comment & remove no-longer used code
* Fix computation errors
* Add heuristic to override num_splits option
* Add constraint to kMaxSplits
* Fix compilation error
* Clean up pipeline code
* Wrap pointer access as lambda function
* Rename confusing methods
* Use kLogMasSplits as template parameter
* Finish splitkv combine kernel codegen
* Update kMaxSplits limit
* Use smaller kM0 for splitkv combine kernel
* Ignore droupout flag in splitkv pipeline
* Unify flag usage
* Add back flag kStoreLSE
* Merge lambda calls in pipeline
* Fix compilation errors
* Avoid all empty splits
* Always check for empty loop in splitkv pipelines
* Re-order parameters
* Remove redundant p_drop option check
* Add traits/problem for fwd splitkv kernel
* Conditionally enable uneven split boundary checks
* Add comment for the splitkv traits field
* Change even split criteria
* Re-order statements
* Refine occupancy value for hdim=128&256
* Refine occupancy value for hdim=32&64
* Remove redundant kernel argument
* Separate fmha bwd codegen logics
* Separate fmha fwd codegen logics
* Remove redundant direction parameter in fwd&bwd codegen logics
* Support generate multiple APIs for an example
* Let 'api' an alias of 'direction' option
* Remove choices for the 'direction' option
* Use dictionary to config all the functions
* Move fmha splitkv codegen logics to other file
* Add fwd_splitkv api for tile_example_fmha_fwd
---------
Co-authored-by: danyao12 <danyao12>
Co-authored-by: carlushuang <carlus.huang@amd.com>
Co-authored-by: rocking <ChunYu.Lai@amd.com>
Co-authored-by: Jing Zhang <jizhan@amd.com>
|
2024-06-26 17:41:15 +08:00 |
|