1.6 KiB
🔀 #225 - Examples : Add new sweep-bench benchmark
| Author | saood06 |
|---|---|
| State | ❌ Closed |
| Created | 2025-02-23 |
| Updated | 2025-04-26 |
Description
Port of 9488fbf1e4
This is a good tool to benchmark with as requested by #223.
As a very quick demo I generated this, just by running this ( ./llama-sweep-bench -c 2048 -ub 512 -m WizardLM-2-8x22B-IQ4_K_R4.gguf -ctk q8_KV -ctv q8_0 -fa --output-format jsonl and then sweep-bench-plot.py with the output).
- Self-reported review complexity:
- Low
- Medium
- High
💬 Conversation
👤 ikawrakow submitted a review the 2025-02-23 at 06:00:18: ✅ APPROVED
Thank you for this - can be very useful.
👤 ubergarm commented the 2025-04-26 at 18:01:12:
@saood06 thanks I'm a convert to llama-sweep-bench! It is indeed very useful.
I pushed a branch on my personal mainline llama.cpp fork just to use for testing performance across forks. I don't plan to open a PR to mainline, but just left it up there in case anyone else is using it. I'm guessing ik has something similar as we were comparing the new GLM-4 performance.
Thanks!