[GH-ISSUE #237] Bug: Inference Runtime downloads produce an empty build folder for CUDA variants (asset naming mismatch with upstream llama.cpp releases) #148

New issue

Open

opened 2026-10-18 14:06:30 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-18 14:06:30 +00:00

Owner

Originally created by @lollo78 on GitHub (Jul 5, 2026).
Original GitHub issue: https://github.com/open-webui/desktop/issues/237

Summary

When Open WebUI Desktop's Inference Runtime checks for a new llama.cpp build, it correctly creates the versioned build folder locally, but the actual binary download silently fails — the folder is created and remains empty. No error is surfaced in the UI; the runtime status stays stuck on "Setting Up".

Suspected root cause

The CUDA variant selector in Settings → Inference Runtime still references CUDA 13.1, which no longer exists as a published asset naming scheme upstream. llama.cpp (ggml-org) has since moved to CUDA 13.3 for the "CUDA 13" Windows variant. Current upstream release assets (example from tag b9873) are named:

llama-b9873-bin-win-cuda-13.3-x64.zip (binary)
cudart-llama-bin-win-cuda-13.3-x64.zip (required CUDA runtime DLLs)

If Open WebUI Desktop's downloader is still constructing/matching asset filenames against the old 13.1 string, the request against the actual release asset will not resolve, causing the download step to fail without any visible error — leaving the newly created build folder empty.

This appears related to but distinct from #198 (which is about missing CUDA DLLs specifically). In this case the entire binary is missing, not just the DLLs.

Steps to reproduce

Open Settings → Inference Runtime.
Set Version to latest (or a recent tag).
Select the CUDA variant intended for NVIDIA GPU acceleration (only CUDA 13.1 is available; missing CUDA 13.3).
Click Start / trigger a build check.
Observe: a new folder is created under the app's llama.cpp builds directory, named after the release tag, but it contains no files.

Expected behavior

The selected CUDA variant should resolve to the correct, currently-published asset name for that llama.cpp release, download both the binary and the associated CUDA runtime DLLs, and populate the build folder correctly.

Actual behavior

Build folder is created (confirming the version-check step works).
No files are downloaded into it (confirming the asset-fetch step fails silently).
Runtime status remains stuck on "Setting Up" with no error message shown to the user.

Environment

OS: Windows 11
Open WebUI Desktop v0.0.20
Inference Runtime Variant selected: CUDA 13.1 (as labeled in current UI dropdown)
GPU: NVIDIA (RTX 5070)

Suggested fix

Dynamically resolve the correct CUDA variant asset name from the GitHub Releases API for the selected llama.cpp tag, instead of hardcoding a fixed CUDA version string (13.1) in the variant-to-asset mapping.
Surface a visible error/toast in the UI when an expected release asset cannot be found, instead of leaving the build folder silently empty.
Update the CUDA variant dropdown to reflect the currently available options (CUDA 13.3 instead of/alongside 13.1) and keep it in sync with upstream naming changes going forward.

Workaround (for anyone hitting this in the meantime)

Manually download the correct binary + cudart-* DLL zip from the llama.cpp release page for the desired tag and extract both into the (empty) build folder the app already created, then click Start instead of Re-download. Pinning Version to a fixed tag (rather than latest) prevents the app from re-triggering the broken download on every launch.

Originally created by @lollo78 on GitHub (Jul 5, 2026). Original GitHub issue: https://github.com/open-webui/desktop/issues/237 ### Summary When Open WebUI Desktop's **Inference Runtime** checks for a new llama.cpp build, it correctly creates the versioned build folder locally, but the actual binary download silently fails — the folder is created and remains empty. No error is surfaced in the UI; the runtime status stays stuck on "Setting Up". <img width="867" height="230" alt="Image" src="https://github.com/user-attachments/assets/4fbb6949-2a48-4457-b1eb-ef090de0574b" /> ### Suspected root cause The CUDA variant selector in Settings → Inference Runtime still references **CUDA 13.1**, which no longer exists as a published asset naming scheme upstream. llama.cpp (ggml-org) has since moved to **CUDA 13.3** for the "CUDA 13" Windows variant. Current upstream release assets (example from tag `b9873`) are named: - `llama-b9873-bin-win-cuda-13.3-x64.zip` (binary) - `cudart-llama-bin-win-cuda-13.3-x64.zip` (required CUDA runtime DLLs) <img width="1943" height="950" alt="Image" src="https://github.com/user-attachments/assets/b16b559e-d276-49e9-9df6-179235ba798e" /> <img width="578" height="414" alt="Image" src="https://github.com/user-attachments/assets/5b7b3f0f-2746-41a7-b03d-57b5c67c94a5" /> If Open WebUI Desktop's downloader is still constructing/matching asset filenames against the old `13.1` string, the request against the actual release asset will not resolve, causing the download step to fail without any visible error — leaving the newly created build folder empty. This appears related to but distinct from #198 (which is about missing CUDA DLLs specifically). In this case the **entire binary** is missing, not just the DLLs. ### Steps to reproduce 1. Open Settings → Inference Runtime. 2. Set Version to `latest` (or a recent tag). 3. Select the CUDA variant intended for NVIDIA GPU acceleration (only CUDA 13.1 is available; missing CUDA 13.3). 4. Click Start / trigger a build check. 5. Observe: a new folder is created under the app's llama.cpp builds directory, named after the release tag, but it contains no files. ### Expected behavior The selected CUDA variant should resolve to the correct, currently-published asset name for that llama.cpp release, download both the binary and the associated CUDA runtime DLLs, and populate the build folder correctly. ### Actual behavior - Build folder is created (confirming the version-check step works). - No files are downloaded into it (confirming the asset-fetch step fails silently). - Runtime status remains stuck on "Setting Up" with no error message shown to the user. ### Environment - OS: Windows 11 - Open WebUI Desktop v0.0.20 - Inference Runtime Variant selected: CUDA 13.1 (as labeled in current UI dropdown) - GPU: NVIDIA (RTX 5070) ### Suggested fix - Dynamically resolve the correct CUDA variant asset name from the GitHub Releases API for the selected llama.cpp tag, instead of hardcoding a fixed CUDA version string (`13.1`) in the variant-to-asset mapping. - Surface a visible error/toast in the UI when an expected release asset cannot be found, instead of leaving the build folder silently empty. - Update the CUDA variant dropdown to reflect the currently available options (CUDA 13.3 instead of/alongside 13.1) and keep it in sync with upstream naming changes going forward. ### Workaround (for anyone hitting this in the meantime) Manually download the correct binary + `cudart-*` DLL zip from the llama.cpp release page for the desired tag and extract both into the (empty) build folder the app already created, then click Start instead of Re-download. Pinning Version to a fixed tag (rather than `latest`) prevents the app from re-triggering the broken download on every launch.

bsakaguchi commented

2026-10-18 14:06:31 +00:00

Author

Owner

@Mamoon-B commented on GitHub (Aug 8, 2026):

I resolved this by

installing ollama on my machine (windows)
copying everything from AppData\Local\Programs\Ollama\lib\ollama\cuda_v13
- Note: If you installed for all users, it will be found in Program Files or Program Files x86.
into open-webui's llama.cpp directory AppData\Roaming\open-webui\llama.cpp\b10326
- Note: b10326 is my current version.
in Open-Webui -> Settings -> Inference Runtime -> Variant -> Auto
Restart lamma-server.

nvidia GPU is now loading models instead of RAM.

@Mamoon-B commented on GitHub (Aug 8, 2026): I resolved this by 1) installing ollama on my machine (windows) 2) copying everything from AppData\Local\Programs\Ollama\lib\ollama\cuda_v13 - Note: If you installed for all users, it will be found in Program Files or Program Files x86. 3) into open-webui's llama.cpp directory AppData\Roaming\open-webui\llama.cpp\b10326 - Note: b10326 is my current version. 4) in Open-Webui -> Settings -> Inference Runtime -> Variant -> Auto 5) Restart lamma-server. nvidia GPU is now loading models instead of RAM.

bsakaguchi referenced this issue

2026-10-18 14:07:03 +00:00

[PR #148] [MERGED] 0.0.10 #215