DocShotgun 156b74f3f0 Revision to paged attention checks (#133) ...

* Model: Clean up paged attention checks

* Model: Move cache_size checks after paged attn checks
Cache size is only relevant in paged mode

* Model: Fix no_flash_attention

* Model: Remove no_flash_attention
Ability to use flash attention is auto-detected, so this flag is unneeded. Uninstall flash attention to disable it on supported hardware.

2024-06-09 17:28:11 +02:00

..

auth.py

Endpoints: Add key permission checker

2024-03-18 00:53:27 -04:00

chat_completion.py

feat: list support in message.content (#122)

2024-06-03 19:57:15 +02:00

common.py

OAI: Add validation to "n"

2024-05-28 00:52:30 -04:00

completion.py

API: Fix finish_reason returns

2024-03-18 15:59:28 -04:00

download.py

Downloader: Fix handling of include pattern

2024-04-30 01:13:06 -04:00

lora.py

Revert "API: Remove unncessary Optional signatures"

2024-05-02 21:23:48 -04:00

model.py

Revision to paged attention checks (#133)

2024-06-09 17:28:11 +02:00

sampler_overrides.py

API: Add preset listing for sampler overrides

2024-05-12 01:34:51 -04:00

template.py

API: Split into separate folder

2024-03-12 23:59:30 -04:00

token.py

OAI: Tokenize chat completion messages

2024-04-15 14:17:16 -04:00