DocShotgun 85387d97ad Fix disabling flash attention in exl2 config (#136) ...

* Model: Fix disabling flash attention in exl2 config

* Model: Pass no_flash_attn to draft config

* Model: Force torch flash SDP off in compatibility mode

2024-06-12 20:00:46 +02:00

..

grammar.py

Allow either "[" or "{" prefix to support JSON grammar with top level arrays (#129)

2024-06-04 02:32:39 +02:00

model.py

Fix disabling flash attention in exl2 config (#136)

2024-06-12 20:00:46 +02:00

utils.py

Revision to paged attention checks (#133)

2024-06-09 17:28:11 +02:00