diff --git a/example/ck_tile/38_block_scale_gemm/README.md b/example/ck_tile/38_block_scale_gemm/README.md index b7b14f9d13..496697ca32 100644 --- a/example/ck_tile/38_block_scale_gemm/README.md +++ b/example/ck_tile/38_block_scale_gemm/README.md @@ -4,9 +4,18 @@ This folder contains examples of quant GEMMs using the ck_tile tile-programming - AQuant kernel with blocks of A matrix sharing scales: custom GEMM pipeline - BQuant kernel with blocks of B matrix sharing scales: custom GEMM pipeline -- Row and Column-wise scaled: All of the rowwise elements in A Matrix and columwise elements in B Matrix will share the same quantization element and the elementwisde operation will complete in epilogue. +- Row and Column-wise scaled: All of the row-wise elements in A Matrix and column-wise elements in B Matrix will share the same quantization element and the element-wise operation will complete in epilogue. - Tensor-wise scaled: Share the same scalar scale across the whole tensor of A or B +## Quantization Mode Comparison + +| Quant Mode | A Matrix Organization | A Scale Shape | B Matrix Organization | B Scale Shape | +|------------|----------------------|---------------|----------------------|---------------| +| **AQuant** | Blocks along K dimension
Each M×GroupSize block shares one scale | `[M, K/GroupSize]` | Not quantized | N/A | +| **BQuant** | Not quantized | N/A | Blocks along K dimension
Each GroupSize×N block shares one scale | `[K/GroupSize, N]` | +| **RowColQuant** | Per-row quantization
All K elements in each row share one scale | `[M, 1]` | Per-column quantization
All K elements in each column share one scale | `[1, N]` | +| **TensorQuant** | Tensor-wise quantization
All M×K elements share one scale | `[1]` | Tensor-wise quantization
All K×N elements share one scale | `[1]` | + --- ## Features