mirror of
https://github.com/SillyTavern/SillyTavern-Extras.git
synced 2026-10-16 02:21:52 +00:00
[GH-ISSUE #18] [Question] Any way to use a second GPU #14
Labels
No labels
bug
duplicate
planned
pull-request
under consideration
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
SillyTavern/SillyTavern-Extras#14
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Manimap on GitHub (May 6, 2023).
Original GitHub issue: https://github.com/SillyTavern/SillyTavern-Extras/issues/18
Hello,
I have my main gpu being totally used by loading a model, but I have a second gpu available locally : is it possible to ask the model to use the second gpu for all requests?
If it's possible, is it doable on windows?
@Cohee1207 commented on GitHub (May 6, 2023):
On line 134 of server.py, change the string of device from cuda:0 to cuda
Please keep in mind that some modules will not work with multi-GPU setup and require a device index. If you encounter it - hard code the device passed to the model initialization to cuda:0
@Manimap commented on GitHub (May 7, 2023):
Alright, I will try that, thank you.