ik_llama.cpp/examples/server/tests/features/server.feature at 917dc8cfa67a72fb7c8bf7392270da3bf4833af4

mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-03-13 15:30:03 +00:00

Files

jaime-m-p 917dc8cfa6 Tokenizer SPM fixes for phi-3 and llama-spm (#7375 )

* Update brute force test: special tokens
* Fix added tokens
  - Try to read 'added_tokens.json'.
  - Try to read 'tokenizer_config.json'.
  - Try to read 'tokenizer.json'.
* Fix special tokens rtrim

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* server : fix test regexes

2024-05-20 20:15:57 +02:00

4.9 KiB

Raw Blame History

View Raw

4.9 KiB Raw Blame History

4.9 KiB

Raw Blame History