q6_0: slightly better kv-cache result

Better than q8_0+q4_0, but not as good as q8_0+iq4_nl
This commit is contained in:
Iwan Kawrakow
2024-10-02 12:02:57 +03:00
parent 9e63f811e1
commit a4b41b4870
2 changed files with 12 additions and 1 deletions

View File

@@ -327,6 +327,9 @@ static ggml_type ggml_type_from_name(const std::string & s) {
if (s == "iq4_nl") {
return GGML_TYPE_IQ4_NL;
}
if (s == "q6_0") {
return GGML_TYPE_Q6_0;
}
return GGML_TYPE_COUNT;
}