kingbri 078fbf1080 Model: Add quantized cache support for tensor parallel ...

Newer versions of exl2 v1.9-dev have quantized cache implemented. Add
those APIs.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-08-22 14:15:19 -04:00

..

grammar.py

Model: Grammar: Fix lru_cache clear function

2024-07-08 15:10:15 -04:00

model.py

Model: Add quantized cache support for tensor parallel

2024-08-22 14:15:19 -04:00

utils.py

Start: Fix pip update, method calls, and logging

2024-08-04 10:30:26 -04:00