turboderp 8d9fd249c6 model_diff.py: Add per-token eval

2026-07-05 14:17:01 +02:00

..

eval_texts

Add simple long-context evaluation script

2025-05-17 16:58:12 +02:00

prompts

SD eval: Add trivial/sanity test

2026-05-26 00:03:39 +02:00

spec

compare_q.py: Add option to format test dataset with chat template (stable measurements for Gemma4)

2026-04-10 20:59:43 +02:00

bbeh_mini.py

BBEH eval: Fix results display

2026-04-04 23:29:19 +02:00

compare_q_anyprecision.py

compare_q.py: Scale model init length to eval length

2026-05-11 14:32:57 +02:00

compare_q_exllamav2.py

compare_q.py: Scale model init length to eval length

2026-05-11 14:32:57 +02:00

compare_q_exllamav3.py

compare_q.py: Scale model init length to eval length

2026-05-11 14:32:57 +02:00

compare_q_llamacpp.py

compare_q.py: Add bpw override to plot dequantized models (e.g. NVFP4)

2026-06-05 19:43:45 +02:00

compare_q_logits.py

compare_q_logits.py: Fix bug

2025-06-08 15:37:41 +02:00

compare_q_plot.py

compare_q.py: Add sub-header with dataset info

2026-06-05 14:13:23 +02:00

compare_q_qtip.py

compare_q.py: Scale model init length to eval length

2026-05-11 14:32:57 +02:00

compare_q_transformers.py

compare_q.py: Deduce HF input device from split

2026-06-05 19:43:45 +02:00

compare_q.py

compare_q.py: Reformat dataset display

2026-06-07 13:09:03 +02:00

diversity.py

Update eval scripts

2026-05-19 00:36:54 +02:00

humaneval.py

Update eval scripts

2026-05-19 00:36:54 +02:00

ifbench.py

Update eval scripts

2026-05-19 00:36:54 +02:00

longctx.py

Update eval scripts

2026-05-19 00:36:54 +02:00

mmlu.py

Update eval scripts

2026-05-19 00:36:54 +02:00

model_diff.py

model_diff.py: Add per-token eval

2026-07-05 14:17:01 +02:00

perf.py

Eval: Debug options

2026-05-29 16:23:37 +02:00

ppl.py

Eval: Update torch_dtype -> dtype

2026-05-29 00:34:18 +02:00

prequant_test.py

Eval: Triton kernels to compute gathered logprobs and KL-div without manifesting huge intermediate tensors

2026-05-13 23:48:31 +02:00

spec_decode.py

Update SD benchmark

2026-06-08 18:24:44 +02:00