CK-UA: VGPR-pressure toggles for kv128 probing (all default OFF)

Adds compile-time levers, all guarded and bit-identical to production when
unset, used to characterise why prefill_d128 fp8 fits KV tile 64 but not 128
under the 256-VGPR/wave ceiling (see ua-test-scripts/kv128_vgpr_findings.md):

- UA_PREFILL_D128_BLOCKSIZE (default 64): KV-tile override for probing kv128.
- UA_FA4_INPLACE_DELTA (default 0): drop sp_delta, scale-shift/exp2 in place on
  sp_compute (fmha_alu_D_upd reads only m/l/o_acc/rowsum_p, never raw scores, so
  bit-identical). VGPR-neutral on its own (compiler already reclaims sp_delta).
- UA_FA4_SHARED_SPCOMPUTE (default 0): keep ONE shared fp32 sp_compute + a
  2-slot fp8 P ping-pong instead of a 2-slot union{sp_compute,p}. The deferred
  PV only needs one live fp32 score; this cuts kv128 spills 173 -> 126. (Forces
  in-place delta; slightly regresses kv64 so it is a kv128-only lever.)
- UA_FA4_UNION_KV (default 0): union k_tile/v_tile (ASM-style). VGPR-neutral;
  kept as a documented dead end (compiler already overlaps their live ranges).

P thread-buffer size exposed as a type-derived constexpr (kPThreadBufSize) so
the static_assert/static_for sites work when sp(idx) is the runtime proxy.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
juuso-oskari
2026-06-11 15:29:04 +00:00
parent 9aa380e6c2
commit 8abbd21a01
2 changed files with 102 additions and 8 deletions

View File

@@ -149,7 +149,14 @@ struct variant_config<KernelVariant::prefill_d128>
// spills at kv64. Unlocking kv128 needs a VGPR-pressure cut (smaller kBlockM
// for this tile, or sub-tiling kBlockN so the live score tile stays 32x64),
// not an LDS change. Stay at 64 until that lands.
static constexpr index_t BlockSize = 64;
//
// UA_PREFILL_D128_BLOCKSIZE: compile-time override of the KV tile so the
// VGPR-pressure experiments (kv128 + sub-tiling) can be probed without
// editing this line each build. Defaults to the production 64.
#ifndef UA_PREFILL_D128_BLOCKSIZE
#define UA_PREFILL_D128_BLOCKSIZE 64
#endif
static constexpr index_t BlockSize = UA_PREFILL_D128_BLOCKSIZE;
using BlockWarps = sequence<8, 1, 1>;
using WarpGemmShape = sequence<32, 32, 16>;
template <typename Problem, index_t PageSize = 0, bool IsPaged = true>