kingbri 2913ce29fc API: Add timings to usage stats ...

It's useful for the client to know what the T/s and total time for
generation are per-request.

Works with both completions and chat completions.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>

2025-06-17 22:54:51 -04:00

..

exllamav2

API: Add timings to usage stats

2025-06-17 22:54:51 -04:00

exllamav3

API: Add timings to usage stats

2025-06-17 22:54:51 -04:00

infinity

Model: Add proper jobs cleanup and fix var calls

2025-04-24 21:30:55 -04:00

base_model_container.py

Model: Create universal HFModel class

2025-05-13 18:12:38 -04:00