deepseek-ai/DeepSeek-VL2
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-VL2.git synced 2026-11-05 01:07:23 +00:00
Code Issues 123 Projects Packages Wiki Activity

[PR #91] Fix the model parallelism error in DeepSeek-VL2 #143

New issue
Open
opened 2026-10-26 21:12:38 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-26 21:12:38 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepSeek-VL2/pull/91
Author: @DeadLining
Created: 2/20/2025
Status: 🔄 Open

Base: main ← Head: deadline

📝 Commits (1)

  • 8ed2d7f Fix the model parallelism error in DeepSeek-VL2

📊 Changes

1 file changed (+10 additions, -0 deletions)

View changed files

📝 deepseek_vl2/models/modeling_deepseek.py (+10 -0)

📄 Description

Fix Model Parallelism Error: KVCache CUDA Device Switching Issue

The RuntimeError:
Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! (when checking argument for argument tensors in method wrapper_CUDA_cat)
error mentioned in #issues8 was caused by KVCache failing to switch CUDA devices correctly during inference.

This change is intended to fix the issue by ensuring proper device switching during inference.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepSeek-VL2/pull/91 **Author:** [@DeadLining](https://github.com/DeadLining) **Created:** 2/20/2025 **Status:** 🔄 Open **Base:** `main` ← **Head:** `deadline` --- ### 📝 Commits (1) - [`8ed2d7f`](https://github.com/deepseek-ai/DeepSeek-VL2/commit/8ed2d7ff8d6bffc25342f578604c1ae358b90e23) Fix the model parallelism error in DeepSeek-VL2 ### 📊 Changes **1 file changed** (+10 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `deepseek_vl2/models/modeling_deepseek.py` (+10 -0) </details> ### 📄 Description Fix Model Parallelism Error: KVCache CUDA Device Switching Issue The RuntimeError: `Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! (when checking argument for argument tensors in method wrapper_CUDA_cat)` error mentioned in [#issues8](https://github.com/deepseek-ai/DeepSeek-VL2/issues/8#issuecomment-2563273225) was caused by KVCache failing to switch CUDA devices correctly during inference. This change is intended to fix the issue by ensuring proper device switching during inference. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-VL2#143
No description provided.