Export ROCm/rocm-libraries@2d4a3223cb

example/ck_tile/10_rmsnorm2d/README.md

@@ -0,0 +1,87 @@
+# RMSNorm2D Forward with CK Tile
+
+This example demonstrates 2D Root Mean Square Layer Normalization (RMSNorm) using the CK Tile programming model, a normalization technique widely used in LLMs and transformers.
+
+---
+
+## Algorithm and Math
+
+For each row $x$:
+$$
+\text{rms}(x) = \sqrt{\frac{1}{N} \sum_{i=1}^N x_i^2 + \epsilon}
+$$
+$$
+y_i = \frac{x_i}{\text{rms}(x)} \cdot \gamma_i
+$$
+where $\gamma$ is a learnable scale parameter.
+
+- **Tilewise RMSNorm**: Each thread block processes a tile (row or block), computes the mean square, normalizes, and applies scale.
+
+---
+
+## Tile Programming Model
+
+- **Tiles**: Each thread block processes a tile of the input matrix.
+- **Pipeline**: Modular, can be extended for fused operations.
+
+---
+
+## Build & Run
+
+```bash
+# in the root of ck_tile
+mkdir build && cd build
+sh ../script/cmake-ck-dev.sh  ../ <arch>  # you can replace this <arch> to gfx90a, gfx942...
+make tile_rmsnorm2d_fwd -j`nproc`
+```
+
+This will result in an executable `build/bin/tile_rmsnorm2d_fwd`
+
+### Arguments
+
+```bash
+args:
+           -m    m dimension (default:3328)
+           -n    n dimension (default:4096)
+    -x_stride    x row_stride, if -1 then equal to n (default:-1)
+   -xr_stride    x residule row_stride, if -1 then equal to n (default:-1)
+    -y_stride    y row_stride, if -1 then equal to n (default:-1)
+   -yr_stride    y residule row_stride, if -1 then equal to n (default:-1)
+           -e    epsilon (default:1e-5)
+    -save_rms    save rms(invrms) or not. set to 1 in training case (default:0)
+-save_unquant    save result before quant (default:0)
+           -v    cpu validation or not (default:1)
+       -kname    print kernel name or not (default:1)
+      -prec_i    input precision (default:fp16)
+      -prec_o    output precision, set auto will be the same as input (default:auto)
+     -prec_sm    output quant scale type, set auto will use fp32. used when fquant=1 (default:auto)
+     -prec_sy    output quant scale type, set auto will use fp32. used when fquant=1 or 2 (default:auto)
+        -fadd    fused-add, 0:no fused add, 1:preadd+store, 2:preadd only (default:0)
+      -fquant    fused-quant, 0:no, 1:smooth-dynamic-quant, 2:dynamic-quant (default:0)
+      -warmup    cold iter (default:5)
+      -repeat    hot iter (default:20)
+           -s    sensitive model mode, 0: for no specific model, 1: for T5-like model (default:0)
+        -json    0: No Json, 1: Dump Results in Json format (default:0)
+    -jsonfile    json file name to dump results (default:rmsnorm2d_fwd.json)
+```
+
+---
+
+## Source Structure
+
+- **Kernel**: [`rmsnorm2d_fwd.hpp`](rmsnorm2d_fwd.hpp) (tile-programming kernel template)
+- **Executable**: [`rmsnorm2d_fwd.cpp`](rmsnorm2d_fwd.cpp) (argument parsing, kernel launch)
+- **Build**: `CMakeLists.txt`, `generate.py`, `script/`
+
+---
+
+## Related CK Tile Examples
+
+- [02_layernorm2d](../02_layernorm2d/README.md): LayerNorm2D with tiles
+- [12_smoothquant](../12_smoothquant/README.md): SmoothQuant with tiles
+- [05_reduce](../05_reduce/README.md): Reductions with tiles
+
+For distribution, see  [`include/ck_tile/tile_program/tile_distribution/`](../../../include/ck_tile/tile_program/tile_distribution/).
+
+---
+[Back to CK Tile Examples](../README.md)