kingbri 09a4c79847 Model: Auto-scale max_tokens by default ...

If max_tokens is None, it automatically scales to fill up the context.
This does not mean the generation will fill up that context since
EOS stops also exist.

Originally suggested by #86

Signed-off-by: kingbri <bdashore3@proton.me>

2024-03-18 22:54:59 -04:00

..

args.py

Tree: Format

2024-03-13 00:02:55 -04:00

auth.py

API: Cleanup permission endpoint

2024-03-18 15:13:26 -04:00

concurrency.py

API: Fix blocking iterator execution

2024-03-16 23:23:31 -04:00

config.py

Tree: Update to cleanup globals

2024-03-12 23:59:30 -04:00

gen_logging.py

Tree: Format

2024-03-13 23:33:18 -04:00

logger.py

Logging: Escape rich markup sequences

2024-03-11 00:28:48 -04:00

model.py

Model: Fix load if model didn't load properly

2024-03-16 23:23:31 -04:00

sampling.py

Model: Auto-scale max_tokens by default

2024-03-18 22:54:59 -04:00

signals.py

Signal: Fix signal handlers for uvicorn

2024-03-16 23:23:31 -04:00

templating.py

Tree: Format

2024-03-13 00:02:55 -04:00

utils.py

Tree: Switch to async generators

2024-03-16 23:23:31 -04:00