mirror of
https://github.com/ROCm/composable_kernel.git
synced 2026-07-16 08:44:55 +00:00
CK-UA: VGPR-pressure toggles for kv128 probing (all default OFF)
Adds compile-time levers, all guarded and bit-identical to production when
unset, used to characterise why prefill_d128 fp8 fits KV tile 64 but not 128
under the 256-VGPR/wave ceiling (see ua-test-scripts/kv128_vgpr_findings.md):
- UA_PREFILL_D128_BLOCKSIZE (default 64): KV-tile override for probing kv128.
- UA_FA4_INPLACE_DELTA (default 0): drop sp_delta, scale-shift/exp2 in place on
sp_compute (fmha_alu_D_upd reads only m/l/o_acc/rowsum_p, never raw scores, so
bit-identical). VGPR-neutral on its own (compiler already reclaims sp_delta).
- UA_FA4_SHARED_SPCOMPUTE (default 0): keep ONE shared fp32 sp_compute + a
2-slot fp8 P ping-pong instead of a 2-slot union{sp_compute,p}. The deferred
PV only needs one live fp32 score; this cuts kv128 spills 173 -> 126. (Forces
in-place delta; slightly regresses kv64 so it is a kv128-only lever.)
- UA_FA4_UNION_KV (default 0): union k_tile/v_tile (ASM-style). VGPR-neutral;
kept as a documented dead end (compiler already overlaps their live ranges).
P thread-buffer size exposed as a type-derived constexpr (kPThreadBufSize) so
the static_assert/static_for sites work when sp(idx) is the runtime proxy.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -149,7 +149,14 @@ struct variant_config<KernelVariant::prefill_d128>
|
||||
// spills at kv64. Unlocking kv128 needs a VGPR-pressure cut (smaller kBlockM
|
||||
// for this tile, or sub-tiling kBlockN so the live score tile stays 32x64),
|
||||
// not an LDS change. Stay at 64 until that lands.
|
||||
static constexpr index_t BlockSize = 64;
|
||||
//
|
||||
// UA_PREFILL_D128_BLOCKSIZE: compile-time override of the KV tile so the
|
||||
// VGPR-pressure experiments (kv128 + sub-tiling) can be probed without
|
||||
// editing this line each build. Defaults to the production 64.
|
||||
#ifndef UA_PREFILL_D128_BLOCKSIZE
|
||||
#define UA_PREFILL_D128_BLOCKSIZE 64
|
||||
#endif
|
||||
static constexpr index_t BlockSize = UA_PREFILL_D128_BLOCKSIZE;
|
||||
using BlockWarps = sequence<8, 1, 1>;
|
||||
using WarpGemmShape = sequence<32, 32, 16>;
|
||||
template <typename Problem, index_t PageSize = 0, bool IsPaged = true>
|
||||
|
||||
Reference in New Issue
Block a user