mirror of https://github.com/ROCm/composable_kernel.git synced 2026-05-01 12:11:19 +00:00

Files

carlushuang 36c7ce4e0e [CK_TILE]Moe update index (#1672 )

* update MOCK_ID for moe-sorting

* add moe-smoothquant

* update a comment

* fix format

* hot fix

* update topk in overflow case

* update comments

* update bf16 cvt

---------

Co-authored-by: valarLip <340077269@qq.com>

2024-11-25 13:12:35 +08:00

664 B

Raw Blame History

moe-smoothquant

This folder contains example for moe-smoothquant using ck_tile tile-programming implementation.

Unlike standard smoothquant op, the input scale is from different expert [expert, hidden], we need reuse the topk-id from previous topk-softmax and select the corresponding expert from current topk, and expand the output/per-token-scale by topk

build

# in the root of ck_tile
mkdir build && cd build
sh ../script/cmake-ck-dev.sh  ../ <arch>  # you can replace this <arch> to gfx90a, gfx942...
make tile_example_moe_smoothquant -j

This will result in an executable build/bin/tile_example_moe_smoothquant

664 B Raw Blame History

moe-smoothquant

build

664 B

Raw Blame History