turboderp-org/exllamav3
1
0
Fork 0
mirror of https://github.com/turboderp-org/exllamav3.git synced 2026-07-12 10:07:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,191 Commits 3 Branches 43 Tags
dev
Commit Graph

104 Commits

Author SHA1 Message Date
turboderp
6af187349e ppl.py: Set dequant mode for MXFP4 tensors in HF path, explicitly move input IDs to model device 2026-07-10 15:01:49 +02:00
turboderp
91404f4e5f MLP: Enable BC/graph path without fused up/gate proj (int8 mode) 2026-07-09 20:19:01 +02:00
turboderp
84c390052b Cache: More efficient quantized packing, add cubic compander to approximate Lloyd-Max, cache quant test scripts and simulated quantization 2026-07-08 16:38:00 +02:00
turboderp
8d9fd249c6 model_diff.py: Add per-token eval 2026-07-05 14:17:01 +02:00
turboderp
db212a692f Update SD benchmark 2026-06-08 18:24:44 +02:00
turboderp
3d5e9e5355 compare_q.py: Reformat dataset display 2026-06-07 13:09:03 +02:00
turboderp
509a6ef0f7 compare_q.py: Deduce HF input device from split 2026-06-05 19:43:45 +02:00
turboderp
21f4302ce6 compare_q.py: Add bpw override to plot dequantized models (e.g. NVFP4) 2026-06-05 19:43:45 +02:00
turboderp
19973ac65a compare_q.py: Clamp logits (lcpp returns -inf for masked tokens) 2026-06-05 18:34:17 +02:00
turboderp
858b695aa0 compare_q.py: Add sub-header with dataset info 2026-06-05 14:13:23 +02:00
turboderp
2e6f069641 compare_q.py: Fix dataset definitions and allow parquet sources 2026-06-05 14:01:51 +02:00
turboderp
5ea0431638 compare_q.py: Stream rows of logits to disks to limit system RAM pressure 2026-06-05 14:00:42 +02:00
turboderp
e9ebaa8b86 compare_q.py: Fix Transformers model inspection, tolerate regression in gptqmodel 2026-06-05 13:59:13 +02:00
turboderp
4b86394080 Eval: Un-disable SD benchmarks 2026-05-29 22:12:45 +02:00
turboderp
6e973a250a Eval: Debug options 2026-05-29 16:23:37 +02:00
turboderp
f3e9ba7ca4 Eval: Update torch_dtype -> dtype 2026-05-29 00:34:18 +02:00
turboderp
60c9949da2 SD eval: Add trivial/sanity test 2026-05-26 00:03:39 +02:00
turboderp
5bb8f97232 compare_q.py: Fix label position 2026-05-25 23:55:48 +02:00
turboderp
30616a4b14 Update eval scripts 2026-05-19 00:36:54 +02:00
turboderp
7a149778fb Eval: Triton kernels to compute gathered logprobs and KL-div without manifesting huge intermediate tensors 2026-05-13 23:48:31 +02:00
turboderp
785551b071 compare_q.py: Clearer border labels for plot and cleaner layout 2026-05-13 23:46:38 +02:00
turboderp
8438123261 EXL3: Increase auto reconstruct threshold 2026-05-12 23:59:10 +02:00
turboderp
db77c9cc01 compare_q.py: More dataset options 2026-05-11 19:23:51 +02:00
turboderp
365e87d1f1 compare_q.py: Fix AutoRound quant layer class issues 2026-05-11 14:34:00 +02:00
turboderp
c2a0e0ddf8 compare_q.py: Fix colors in plot for select quant types 2026-05-11 14:33:25 +02:00
turboderp
3d0f9c5bbb compare_q.py: Scale model init length to eval length 2026-05-11 14:32:57 +02:00
turboderp
275d62841b compare_q.py: Fix bitrate measurement for multimodal HF architectures, measure native torch.nn layers with packed weights 2026-05-11 01:08:37 +02:00
turboderp
6645b74c40 compare_q.py: Swap matplotlib for seaborn, new plot layout engine 2026-05-10 20:48:56 +02:00
turboderp
ad098d6cbe compare_q.py: Update after latest dependency sabotage 2026-05-10 17:52:55 +02:00
turboderp
9a3041c5b0 longctx eval: Allow draft model 2026-05-09 13:30:22 +02:00
turboderp
11a2a849cc Eval: Add -gp/--gen_prompt option to eval scripts 2026-05-07 20:34:02 +02:00
turboderp
c6a54acbe1 SD benchmark: Reduce default cache size 2026-05-05 10:04:29 +02:00
turboderp
5416d5acc9 spec_decode eval: Allow overriding default num_draft_tokens 2026-04-25 04:00:18 +02:00
turboderp
67ddb6b4ec Add spec_decode eval 2026-04-25 03:20:48 +02:00
turboderp
cb50b9fa6a afmoe: Assert topk_groups=1 and use dots router 2026-04-23 21:05:21 +02:00
turboderp
7f6459f259 MMLU eval: Add redux option 2026-04-18 18:30:25 +02:00
turboderp
0d3893face Loader: Allow specifying max bsz for autosplit to better estimate recurrent state VRAM overhead 2026-04-18 14:12:29 +02:00
turboderp
e90fe55e89 compare_q.py: Add option to format test dataset with chat template (stable measurements for Gemma4) 2026-04-10 20:59:43 +02:00
turboderp
7046a5c739 perf.py: Fix cache overflow 2026-04-06 01:58:27 +02:00
turboderp
514389a2b5 ppl.py: No default datatype for HF mode 2026-04-05 19:43:46 +02:00
turboderp
476ad297ec BBEH eval: Fix results display 2026-04-04 23:29:19 +02:00
turboderp
5bb4e0d32b MMLU eval: Fix confidence interval 2026-04-04 22:49:51 +02:00
turboderp
76acd9c140 Eval: Add BigBench Extra Hard 2026-04-04 22:31:26 +02:00
turboderp
9bd2b5ea4d ppl eval: Combine HF and EXL3 evals into single module, add mode that attempts to replicate default llama.cpp eval tokenization and scoring 2026-04-03 23:37:00 +02:00
turboderp
7d80c39a45 Add IFBench eval 2026-03-25 01:58:45 +01:00
turboderp
85cb54c6f3 perf.py: Make sure test context is nontrivial to force more expert diversity 2026-03-07 01:18:27 +01:00
turboderp
67785fc286 compare_q.py: Paper over some dependency problems 2026-03-02 18:47:39 +01:00
turboderp
b2b6f37e12 perf.py: Error out if test length > cache size 2026-02-17 20:04:13 +01:00
MikeRoz47
52c2f5794d Add optional arg to compare_q to allow it to save plots rather than show them 2026-02-15 16:41:18 +00:00
turboderp
428a082276 Add performance test 2026-01-22 23:28:53 +01:00