deepseek-ai/DeepSeek-V2
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-V2.git synced 2026-11-05 00:47:23 +00:00
Code Issues 90 Projects Packages Wiki Activity

[GH-ISSUE #37] 服务器部署问题 #34

New issue
Open
opened 2026-10-26 21:07:02 +00:00 by bsakaguchi · 2 comments
bsakaguchi commented 2026-10-26 21:07:02 +00:00
Owner
Copy link

Originally created by @airsxue on GitHub (May 17, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/37

请问我用这个demo在linux运行v2需要安装哪些环境？谢谢
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig

model_name = "deepseek-ai/DeepSeek-V2-Chat"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

max_memory should be set based on your devices

max_memory = {i: "75GB" for i in range(8)}

device_map cannot be set to auto

model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="sequential", torch_dtype=torch.bfloat16, max_memory=max_memory, attn_implementation="eager")
model.generation_config = GenerationConfig.from_pretrained(model_name)
model.generation_config.pad_token_id = model.generation_config.eos_token_id

messages = [
{"role": "user", "content": "Write a piece of quicksort code in C++"}
]
input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(input_tensor.to(model.device), max_new_tokens=100)

result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)
print(result)

Originally created by @airsxue on GitHub (May 17, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/37 请问我用这个demo在linux运行v2需要安装哪些环境？谢谢 import torch from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig model_name = "deepseek-ai/DeepSeek-V2-Chat" tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) # `max_memory` should be set based on your devices max_memory = {i: "75GB" for i in range(8)} # `device_map` cannot be set to `auto` model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="sequential", torch_dtype=torch.bfloat16, max_memory=max_memory, attn_implementation="eager") model.generation_config = GenerationConfig.from_pretrained(model_name) model.generation_config.pad_token_id = model.generation_config.eos_token_id messages = [ {"role": "user", "content": "Write a piece of quicksort code in C++"} ] input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt") outputs = model.generate(input_tensor.to(model.device), max_new_tokens=100) result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True) print(result)
bsakaguchi commented 2026-10-26 21:07:03 +00:00
Author
Owner
Copy link

@haichuan1221 commented on GitHub (May 20, 2024):

请问我用这个demo在linux运行v2需要安装哪些环境？谢谢 import torch from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig

model_name = "deepseek-ai/DeepSeek-V2-Chat" tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

max_memory should be set based on your devices

max_memory = {i: "75GB" for i in range(8)}

device_map cannot be set to auto

model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="sequential", torch_dtype=torch.bfloat16, max_memory=max_memory, attn_implementation="eager") model.generation_config = GenerationConfig.from_pretrained(model_name) model.generation_config.pad_token_id = model.generation_config.eos_token_id

messages = [ {"role": "user", "content": "Write a piece of quicksort code in C++"} ] input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt") outputs = model.generate(input_tensor.to(model.device), max_new_tokens=100)

result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True) print(result)

同问

<!-- gh-comment-id:2119556773 --> @haichuan1221 commented on GitHub (May 20, 2024): > 请问我用这个demo在linux运行v2需要安装哪些环境？谢谢 import torch from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig > > model_name = "deepseek-ai/DeepSeek-V2-Chat" tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) > > # `max_memory` should be set based on your devices > max_memory = {i: "75GB" for i in range(8)} > > # `device_map` cannot be set to `auto` > model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="sequential", torch_dtype=torch.bfloat16, max_memory=max_memory, attn_implementation="eager") model.generation_config = GenerationConfig.from_pretrained(model_name) model.generation_config.pad_token_id = model.generation_config.eos_token_id > > messages = [ {"role": "user", "content": "Write a piece of quicksort code in C++"} ] input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt") outputs = model.generate(input_tensor.to(model.device), max_new_tokens=100) > > result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True) print(result) 同问
bsakaguchi commented 2026-10-26 21:07:03 +00:00
Author
Owner
Copy link

@stack-heap-overflow commented on GitHub (May 27, 2024):

需要安装的python库包括：torch，transformers，accelerate。

相关库不同版本的兼容性并没有详细测过，这里可以给一个我测试用的环境供参考，不一定需要严格符合：

  • torch == 2.1.0
  • transformers == 4.39.3
  • accelerate == 0.29.3
<!-- gh-comment-id:2133282958 --> @stack-heap-overflow commented on GitHub (May 27, 2024): 需要安装的python库包括：`torch`，`transformers`，`accelerate`。 相关库不同版本的兼容性并没有详细测过，这里可以给一个我测试用的环境供参考，不一定需要严格符合： - `torch == 2.1.0` - `transformers == 4.39.3` - `accelerate == 0.29.3`
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-V2#34
No description provided.