tarcey e8679910c5 llama : fix typical sampling (#4261) ...

Typical sampling was broken because after copying new_candidates into canditates, the "sorted" bool is left at "true", but the new data is no longer sorted according to probability. Patch to set "sorted" to false.

Test: Generating with temp=0.0001 (approx. argmax)  should generate the same sequence at typical>=1.0 and typical=0.9999 (approx. disabled, but enters the typical sampling codepath).

2023-11-30 23:40:23 +02:00

361 KiB

Raw Blame History

View Raw