mirror of
https://github.com/theroyallab/tabbyAPI.git
synced 2026-03-14 15:57:27 +00:00
67507105d0
* Exposed draft model args for speculative decoding * Exposed int8 cache, dummy models, and no flash attention * Resolved CUDA 11.8 dependency issue
11 lines
368 B
Plaintext
11 lines
368 B
Plaintext
fastapi
|
|
pydantic < 2,>= 1
|
|
PyYAML
|
|
progress
|
|
uvicorn
|
|
|
|
# Wheels
|
|
|
|
# Linux FA2 from https://github.com/Dao-AILab/flash-attention/releases
|
|
https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.6/flash_attn-2.3.6+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.10"
|