CUDA: set compute parameters via command line arguments (#910)

* cuda: set compute parameters via command line arguments * Also llama-bench --------- Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2026-03-12 23:10:01 +00:00 · 2025-11-07 07:11:23 +02:00
parent 49befdd4fb
commit 532a05e466
11 changed files with 144 additions and 29 deletions
--- a/include/llama.h
+++ b/include/llama.h
@@ -439,6 +439,7 @@ extern "C" {
        ggml_abort_callback abort_callback;
        void *              abort_callback_data;
        void *              offload_policy;
+        void *              cuda_params;
    };

    // model quantization parameters