Special handling of Seed Coder FIM tokens (#585)

* Special handling of Seed Coder FIM tokens

* vocab: Add Seed Coder pretokenizer

* Formatting fix

* Update llama.h
This commit is contained in:
Fizz~
2025-07-06 06:13:55 -04:00
committed by GitHub
parent 49d4d2630a
commit 27ff5bf57e
5 changed files with 27 additions and 0 deletions

View File

@@ -110,6 +110,7 @@ extern "C" {
LLAMA_VOCAB_PRE_TYPE_LLAMA4 = 33,
LLAMA_VOCAB_PRE_TYPE_FALCON_3 = 34,
LLAMA_VOCAB_PRE_TYPE_FALCON_E = 35,
LLAMA_VOCAB_PRE_TYPE_SEED_CODER = 36, //llama.cpp lists this as 35
};
// note: these values should be synchronized with ggml_rope