mirror of
https://github.com/ROCm/composable_kernel.git
synced 2026-05-14 02:02:46 +00:00
4fad52fea6
* update MOCK_ID for moe-sorting
* add moe-smoothquant
* update a comment
* fix format
* hot fix
* update topk in overflow case
* update comments
* update bf16 cvt
---------
Co-authored-by: valarLip <340077269@qq.com>
[ROCm/composable_kernel commit: 36c7ce4e0e]
moe-smoothquant
This folder contains example for moe-smoothquant using ck_tile tile-programming implementation.
Unlike standard smoothquant op, the input scale is from different expert [expert, hidden], we need reuse the topk-id from previous topk-softmax and select the corresponding expert from current topk, and expand the output/per-token-scale by topk
build
# in the root of ck_tile
mkdir build && cd build
sh ../script/cmake-ck-dev.sh ../ <arch> # you can replace this <arch> to gfx90a, gfx942...
make tile_example_moe_smoothquant -j
This will result in an executable build/bin/tile_example_moe_smoothquant