turboderp 082a9fe9df Fix Q4 cache in chat example

2024-03-06 19:13:21 +01:00

..

batched_inference.py

Cleanup examples a bit

2024-01-20 10:57:16 +01:00

cfg.py

CFG support in streaming gen

2024-01-01 23:48:24 +01:00

chat_example.ipynb

Add files via upload

2023-11-23 16:02:50 +01:00

chat_formatting.py

Fixed padding issue

2023-11-29 01:43:20 +01:00

chat_prompts.py

Add Gemma prompt format to example chatbot

2024-02-22 14:43:42 +01:00

chat.py

Fix Q4 cache in chat example

2024-03-06 19:13:21 +01:00

inference.py

Cleanup examples a bit

2024-01-20 10:57:16 +01:00

lm_format_enforcer.py

Update examples

2024-03-06 02:41:23 +01:00

lora.py

Remove return values from load_gen

2024-02-14 19:41:59 +01:00

minimal_chat.py

Update examples

2024-03-06 02:41:23 +01:00

multiple_caches.py

Fix multiple caches not working with 8-bit cache mode

2023-12-17 14:41:21 +01:00

speculative.py

Update examples

2024-03-06 02:41:23 +01:00

streaming.py

Update examples

2024-03-06 02:41:23 +01:00

ws_server.py

Update examples (auto GPU split)

2023-10-22 19:32:26 +02:00