kingbri 740b0215dd Model: Dynamically scale generate_window ...

Allows for adjustment of reservation space at the end of the context
before rolling it. This should be scaled as a model's max_seq_len
goes up.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-01-25 00:15:40 -05:00

32 KiB

Raw Blame History

View Raw