13 Commits

Author	SHA1	Message	Date
turboderp	cad7848375	HumanEval: Rename new args to match other scripts	2024-09-29 12:57:06 +02:00
Llama Enjoyer	b2af0bbad3	Remove stray import.	2024-09-24 17:32:09 +02:00
Llama Enjoyer	3a389131de	Add more arguments to accept values passed via the cmd line.	2024-09-24 17:28:02 +02:00
Llama Enjoyer	e960dfd68d	Fix the temperature argument to accept values passed via the cmd line.	2024-09-24 17:18:08 +02:00
turboderp	8dda94fc06	HumanEval: Update templates, add Llama/Mistral template	2024-07-30 11:07:23 +02:00
turboderp	010252aaec	HumanEval: Add temp argument, update default temp	2024-07-30 11:07:03 +02:00
turboderp	0122b1192f	Option to launch eval script automatically after HumanEval test	2024-07-10 02:51:50 +02:00
turboderp	1e31fbf5d3	HumanEval: add Gemma template	2024-07-09 08:05:39 +02:00
turboderp	675450d845	Add Q6 and Q8 cache options to eval scripts	2024-06-09 02:13:06 +02:00
turboderp	4dea0c2451	Shuffle option for MMLU eval	2024-06-06 11:54:25 +02:00
turboderp	64440cff9f	Use standard prompt format for MMLU	2024-06-01 04:39:10 +02:00
turboderp	823bf11c68	Update MMLU test to use dynamic batching	2024-06-01 03:31:40 +02:00
turboderp	e7cbb300ff	Update HumanEval test to dynamic generator	2024-05-31 22:24:16 +02:00