92 Commits

Author	SHA1	Message	Date
turboderp	082a9fe9df	Fix Q4 cache in chat example	2024-03-06 19:13:21 +01:00
turboderp	eb8269726f	Update examples	2024-03-06 02:41:23 +01:00
turboderp	d09f97aedc	Add Q4 option to chat example	2024-03-05 00:29:12 +01:00
turboderp	1de4cdd70b	Add skew sampling	2024-02-25 15:53:31 +01:00
turboderp	69fba75225	Add Gemma prompt format to example chatbot	2024-02-22 14:43:42 +01:00
turboderp	c8e2bf4594	Fix small mistake in example	2024-02-19 14:20:17 +01:00
turboderp	229019d86e	Add lm-format-enforcer JSON example	2024-02-19 00:56:06 +01:00
turboderp	b0dc588d9b	Remove return values from load_gen	2024-02-14 19:41:59 +01:00
turboderp	9f8951e63b	More typeable arg shortcut	2024-02-02 15:03:16 +01:00
Alexander Abushady	8461e6fa76	Kalomaze's Quadratic Sampling ... Quadratic Sampling	2024-02-01 00:11:44 -05:00
turboderp	8c9a3ecb49	Add dyn temp options to chat example	2024-01-30 17:51:59 +01:00
turboderp	0f83192963	Merge branch 'pr-238' ... Change probs return type to tensor	2024-01-20 13:14:21 +01:00
turboderp	99b19ec5f1	Cleanup examples a bit	2024-01-20 10:57:16 +01:00
turboderp	376218f70b	Add batch inference example	2024-01-20 10:51:52 +01:00
turboderp	2ad2a65d3a	Add nous prompt format	2024-01-19 16:45:37 +01:00
turboderp	41b15dd1c3	Refactor to consolidate attn params	2024-01-04 04:52:49 +01:00
turboderp	66d19b6aa9	CFG support in streaming gen	2024-01-01 23:48:24 +01:00
turboderp	addab083b6	Merge pull request #251 from eramax/patch-1 ... add openchat prompt format	2023-12-31 21:27:31 +01:00
turboderp	bdc57362a7	Add minimal chat example	2023-12-31 03:40:56 +01:00
Ahmed Morsi	cf92bcb7ee	add penchat prompt format	2023-12-29 10:59:34 -08:00
turboderp	7a21396854	Merge branch 'feat/frequency_presence_pen' ... # Conflicts: # exllamav2/generator/sampler.py	2023-12-25 18:05:02 +01:00
turboderp	f0c516d7c0	Slight tweaks to SD	2023-12-25 16:54:13 +01:00
Ivan Sanchez	f8afef97f7	return token probabilities in generator made optional. Change generator examples back to default case	2023-12-24 11:17:29 +00:00
turboderp	bf2710f008	Optimizer batched sampling	2023-12-23 22:04:10 +01:00
AlpinDale	a531dea6a0	Merge branch 'turboderp:master' into feat/frequency_presence_pen	2023-12-23 01:42:00 +00:00
AlpinDale	1384eb540a	add frequency and presence penalties	2023-12-21 17:19:47 +00:00
AlpinDale	5131099b5f	add top_a in a few more places	2023-12-21 15:28:34 +00:00
Ivan Sanchez	41efa463cd	unpack prob from return of generator.stream()	2023-12-21 10:54:52 +00:00
turboderp	d1f2952cd6	Fix multiple caches not working with 8-bit cache mode	2023-12-17 14:41:21 +01:00
turboderp	ff2abe38f5	Solar prompt format	2023-12-14 01:28:34 +01:00
turboderp	5c974259bd	More sensible defaults sampling parameters	2023-12-03 22:09:41 +01:00
turboderp	89885be0fe	Merge pull request #204 from SinanAkkoyun/chat-fix ... Added draft model rope scale to chat example	2023-12-03 08:48:19 +01:00
Sinan Akkoyun	81111ee911	Added draft model rope scale	2023-12-03 06:14:49 +00:00
Sinan	cad6b46832	Merge branch 'turboderp:master' into code-chat	2023-12-03 04:29:51 +00:00
turboderp	a9ebe04b0b	Add amnesia option to chatbot	2023-12-01 19:10:58 +01:00
Sinan	f03307baf3	Added DeepSeek Coder Instruct to chat example	2023-11-29 14:44:58 +01:00
SinanAkkoyun	cc1ca45250	Fixed padding issue	2023-11-29 01:43:20 +01:00
SinanAkkoyun	a58c162de7	Fixed out of bounds code block	2023-11-28 20:34:36 +01:00
turboderp	2119668f1e	Add files via upload	2023-11-23 16:02:50 +01:00
turboderp	dfd0bcf888	Revert example	2023-11-22 07:23:43 +01:00
turboderp	5886047b15	Don't update setuptools	2023-11-22 07:07:48 +01:00
turboderp	8e29e009b0	Add Mirostat sampling	2023-11-12 05:29:04 +01:00
turboderp	b302e310c8	More output in SD example	2023-11-10 20:16:08 +01:00
turboderp	d0cadb56a6	Update example	2023-11-01 21:07:33 +01:00
turboderp	f5d99f2e02	Add multi-cache example	2023-11-01 19:09:05 +01:00
turboderp	747a8657ad	Merge pull request #121 from nivibilla/patch-1 ... Fix some syntax errors	2023-10-22 19:54:30 +02:00
turboderp	7a783b3824	Update examples (auto GPU split)	2023-10-22 19:32:26 +02:00
Srinivas Billa	34edcf1947	Fix some syntax errors	2023-10-19 20:08:58 +01:00
turboderp	fb350d76ed	Add 8-bit cache mode to chatbot	2023-10-15 23:16:21 +02:00
turboderp	c2efd2c00c	Apply alpha scaling to draft model when necessary ... Collect some metrics on speculative decoding	2023-10-14 22:30:59 +02:00