[PR #106] [MERGED] Fix: Install correct KoboldCpp CUDA binary for NVIDIA GPU (koboldcpp_cu12.exe) #123

New issue

Closed

opened 2026-10-15 18:07:10 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:07:10 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/SillyTavern/SillyTavern-Launcher/pull/106
Author: @trungly1
Created: 5/26/2025
Status: ✅ Merged
Merged: 5/27/2025
Merged by: @deffcolony

Base: main ← Head: main

📝 Commits (2)

a831e0c Change Koboldcpp URL to cuda
2d5fced Use latest koboldcpp_cu12.exe URL for NVIDIA install instad of hardlink

📊 Changes

2 files changed (+2 additions, -2 deletions)

View changed files

📝 Launcher.bat (+1 -1)
📝 bin/functions/Toolbox/App_Installer/Text_Completion/install_koboldcpp.bat (+1 -1)

📄 Description

Summary

This PR fixes the NVIDIA GPU install logic in the launcher, which previously downloaded koboldcpp.exe — the CPU-only binary. This caused issues when using Flash Attention with GGUF models.

Fix

Replaced the URL with the correct CUDA-enabled binary:

https://github.com/LostRuins/koboldcpp/releases/download/v1.92.1/koboldcpp_cu12.exe

Why It Matters

Downloading the incorrect binary causes a CUDA architecture error at runtime:

ERROR: CUDA kernel flash_attn_ext_f16 has no device code compatible with CUDA arch 750.

This change ensures the App Installer downloads the correct version for NVIDIA users, restoring Flash Attention support.

Tested locally with a GGUF model on RTX 4090.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/SillyTavern/SillyTavern-Launcher/pull/106 **Author:** [@trungly1](https://github.com/trungly1) **Created:** 5/26/2025 **Status:** ✅ Merged **Merged:** 5/27/2025 **Merged by:** [@deffcolony](https://github.com/deffcolony) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (2) - [`a831e0c`](https://github.com/SillyTavern/SillyTavern-Launcher/commit/a831e0cb4907e91d68274c056e633e5da41eeba9) Change Koboldcpp URL to cuda - [`2d5fced`](https://github.com/SillyTavern/SillyTavern-Launcher/commit/2d5fced38b64380d5362c5f07d7dcf98ead53b05) Use latest koboldcpp_cu12.exe URL for NVIDIA install instad of hardlink ### 📊 Changes **2 files changed** (+2 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `Launcher.bat` (+1 -1) 📝 `bin/functions/Toolbox/App_Installer/Text_Completion/install_koboldcpp.bat` (+1 -1) </details> ### 📄 Description ### Summary This PR fixes the NVIDIA GPU install logic in the launcher, which previously downloaded `koboldcpp.exe` — the CPU-only binary. This caused issues when using Flash Attention with GGUF models. ### Fix Replaced the URL with the correct CUDA-enabled binary: `https://github.com/LostRuins/koboldcpp/releases/download/v1.92.1/koboldcpp_cu12.exe` ### Why It Matters Downloading the incorrect binary causes a CUDA architecture error at runtime: ``` ERROR: CUDA kernel flash_attn_ext_f16 has no device code compatible with CUDA arch 750. ``` This change ensures the App Installer downloads the correct version for NVIDIA users, restoring Flash Attention support. Tested locally with a GGUF model on RTX 4090. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-15 18:07:10 +00:00