turboderp 9a3041c5b0 longctx eval: Allow draft model

2026-05-09 13:30:22 +02:00

..

eval_texts

Add simple long-context evaluation script

2025-05-17 16:58:12 +02:00

prompts

Add spec_decode eval

2026-04-25 03:20:48 +02:00

spec

compare_q.py: Add option to format test dataset with chat template (stable measurements for Gemma4)

2026-04-10 20:59:43 +02:00

bbeh_mini.py

BBEH eval: Fix results display

2026-04-04 23:29:19 +02:00

compare_q_anyprecision.py

compare_q.py: Fix/ignore anyprecision imports (Transformers version mismatch)

2025-07-16 09:49:18 +02:00

compare_q_exllamav2.py

compare_q.py: Allow script to run without all backends installed

2025-06-05 02:26:00 +02:00

compare_q_exllamav3.py

compare_q.py: Add option to format test dataset with chat template (stable measurements for Gemma4)

2026-04-10 20:59:43 +02:00

compare_q_llamacpp.py

compare_q.py: Add option to format test dataset with chat template (stable measurements for Gemma4)

2026-04-10 20:59:43 +02:00

compare_q_logits.py

compare_q_logits.py: Fix bug

2025-06-08 15:37:41 +02:00

compare_q_qtip.py

compare_q.py: Add QTIP wrapper

2025-06-08 15:41:30 +02:00

compare_q_transformers.py

compare_q.py: Add option to format test dataset with chat template (stable measurements for Gemma4)

2026-04-10 20:59:43 +02:00

compare_q.py

compare_q.py: Add option to format test dataset with chat template (stable measurements for Gemma4)

2026-04-10 20:59:43 +02:00

diversity.py

Diversity test: use greedy sampling for extraction

2026-01-14 21:40:31 +01:00

humaneval.py

HumanEval: Add max batch size arg

2025-12-05 13:21:07 +01:00

ifbench.py

MMLU eval: Fix confidence interval

2026-04-04 22:49:51 +02:00

longctx.py

longctx eval: Allow draft model

2026-05-09 13:30:22 +02:00

mmlu.py

MMLU eval: Add redux option

2026-04-18 18:30:25 +02:00

model_diff.py

Eval: Add -gp/--gen_prompt option to eval scripts

2026-05-07 20:34:02 +02:00

perf.py

afmoe: Assert topk_groups=1 and use dots router

2026-04-23 21:05:21 +02:00

ppl.py

Eval: Add -gp/--gen_prompt option to eval scripts

2026-05-07 20:34:02 +02:00

prequant_test.py

Eval: Add -gp/--gen_prompt option to eval scripts

2026-05-07 20:34:02 +02:00

spec_decode.py

SD benchmark: Reduce default cache size

2026-05-05 10:04:29 +02:00