imatrix: be able to specify the name of the output tensor

For some models the same tensor is used for token embeddings and
output. This tensor tends to be named token_embedding.weight rather
than output.weight, which prevernts us from collecting imatrix data
for this tensor. With this commit we can tell the name of the
output tensor to the imatrix tool.
This commit is contained in:
Iwan Kawrakow
2024-06-26 17:38:18 +03:00
parent 71725a918f
commit 0a3a2c4cd4
3 changed files with 11 additions and 1 deletions

View File

@@ -1599,6 +1599,14 @@ bool gpt_params_find_arg(int argc, char ** argv, const std::string & arg, gpt_pa
params.process_output = true;
return true;
}
if (arg == "--output-tensor-name") {
if (++i >= argc) {
invalid_param = true;
return true;
}
params.output_tensor_name = argv[i];
return true;
}
if (arg == "--no-ppl") {
params.compute_ppl = false;
return true;