kingbri 8824ea0205 Model: Add EOS token support from generation_config.json ...

GenerationConfig is meant to override various parts of the model
on generation within the transformers lib. Rather than implementing
the entire GenerationConfig framework (since it's pretty redundant),
add in multi eos_token support like VLLM.

The GenerationConfig is used only for generation, but can be used
for other uses if needed.

If there's more necessary parameters in the future, add those in
as well.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-04-19 22:52:32 -04:00

..

grammar.py

Dependencies: Switch to LM-format-enforcer fork

2024-04-14 11:59:49 -04:00

model.py

Model: Add EOS token support from generation_config.json

2024-04-19 22:52:32 -04:00

utils.py

Requirements: Update Exllamav2

2024-04-07 18:00:56 -04:00