ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-02-26 16:14:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

Be able to re-quantize MS BitNet I2_S models (#169)

Browse Source 
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
This commit is contained in:
Kawrakow
2025-01-10 18:18:04 +02:00
committed by GitHub
parent c411615505
commit 400b774294
6 changed files with 58 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
src/llama.cpp
View File

@@ -15698,6 +15698,12 @@ static void llama_tensor_dequantize_internal(
throw std::runtime_error(format("cannot dequantize/convert tensor type %s", ggml_type_name(tensor->type)));
}
if (tensor->type == GGML_TYPE_I2_S) {
// we need to dequantize the entire tensor for I2_S
qtype.to_float(tensor->data, f32_output, nelements);
return;
}
if (nthread < 2) {
if (tensor->type == GGML_TYPE_F16) {
ggml_fp16_to_fp32_row((ggml_fp16_t *)tensor->data, f32_output, nelements);