mb8565 a4e5f7cc44 GLM-DSA: convert sparse-attention control from env vars to CLI args (off by default) ...

Implements ikawrakow's direction from discussion #2040: the DSA sparse
indexer must be controllable via command-line argument (not environment
variables), and must be OFF by default for now.

Control surface, before -> after:
  DSA_INDEXER_DISABLE (env, inverted: on-by-default)  -> --dsa / -dsa
      (cparams.dsa, default false; opt-in, dense-by-default)
  DSA_TOPK_OVERRIDE   (env)                            -> --dsa-top-k N / -dsatk N
      (cparams.dsa_top_k, default -1 == model's configured indexer_top_k)
  DSA_HADAMARD_DISABLE, DSA_SINK (env)                 -> kept as DEBUG-ONLY env
      knobs (clearly commented; no CLI surface, not system on/off controls)

Plumbing mirrors existing boolean/int feature flags (-mla, -khad):
  include/llama.h        llama_context_params {bool dsa; int dsa_top_k;}
  src/llama.cpp          default_params (false / -1); cparams assignment
  src/llama-cparams.h    llama_cparams {bool dsa=false; int dsa_top_k=-1;}
  common/common.h        gpt_params {bool dsa=false; int dsa_top_k=-1;}
  common/common.cpp      arg parse + help text + cparams copy
  src/graphs/build_deepseek2.cpp  gate now checks cparams.dsa instead of
      getenv; top-k override reads cparams.dsa_top_k. Stays arch-gated to
      LLM_ARCH_GLM_DSA. When --dsa is off (default) the indexer function is
      never called -> existing dense MLA path, byte-identical to no-feature.

Validation (GLM-5.2-UD-IQ2_M, 3x P100, -ngl 99 --cpu-moe -mla 3 -fa 1,
wikitext-2, 4 chunks @ c2560):
  --dsa OFF (default, dense):              PPL 2.4151  (graph nodes 4166)
  --dsa ON, default top_k=2048:            PPL 2.4697  (graph nodes 8846)
  --dsa ON, --dsa-top-k 1024:              PPL 3.5107
Off-by-default runs the dense path; ON activates the indexer (node count
jumps, PPL shifts as the top-k mask bites once n_kv > top_k). No env var
is consulted for the primary on/off or the top-k knob.

Graph-parallel (-sm graph) interaction (the item ikawrakow flagged):
Under -sm graph the MLA layers are TP-split (wo->extra) and route to
build_deepseek2_tp_attention(), which contains NO indexer code. So --dsa
is silently a NO-OP under -sm graph: it does not error or crash, it runs
dense. Empirically, --dsa --dsa-top-k 1024 under -sm graph gives
PPL 2.4308 (chunks 1.6967/1.7906/2.1664/2.4308) -- the dense baseline
(2.4151), NOT the DSA top_k=1024 numbers (3.5107). The 0.016 delta is
f16 TP-reduce numerics, not DSA. Conclusion: DSA "works under deepseek2"
only on the non-TP (layer) path; serving DSA with -sm graph would require
wiring the indexer into the TP attention path (or a dedicated DSA arch).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-27 11:46:38 -05:00

..

llama.h

GLM-DSA: convert sparse-attention control from env vars to CLI args (off by default)

2026-06-27 11:46:38 -05:00