deepseek-ai/DeepSeek-V2
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-V2.git synced 2026-11-05 00:47:23 +00:00
Code Issues 90 Projects Packages Wiki Activity

[GH-ISSUE #34] Invalid max_token values #32

New issue
Open
opened 2026-10-26 21:07:00 +00:00 by bsakaguchi · 2 comments
bsakaguchi commented 2026-10-26 21:07:00 +00:00
Owner
Copy link

Originally created by @audreyeternal on GitHub (May 16, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/34

I followed the instruction in README about how to utilize deepseek in langchain:

model = ChatOpenAI(
        model="deepseek-chat",
        openai_api_key=API_KEY,
        openai_api_base='https://api.deepseek.com/v1',
        temperature=0.85,
        max_tokens=8000)

However, seems like the max_tokens is still restricted to 4k and an error will be raised when intending to integrate the model into the chain and invoke it:

qa_with_sources = RetrievalQAWithSourcesChain.from_chain_type(
    llm=model,
    chain_type="stuff",
    retriever=pinecone.as_retriever()
)
query = "foobar"

result = qa_with_sources.invoke(query)

Warning

File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/base.py", line 153, in invoke
self._call(inputs, run_manager=run_manager)
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 137, in _call
output, extra_return_dict = self.combine_docs(
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 244, in combine_docs
return self.llm_chain.predict(callbacks=callbacks, **inputs), {}
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/llm.py", line 316, in predict
return self(kwargs, callbacks=callbacks)[self.output_key]
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 148, in warning_emitting_wrapper
return wrapped(*args, **kwargs)
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/base.py", line 378, in call
return self.invoke(
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/base.py", line 163, in invoke
raise e
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/base.py", line 153, in invoke
self._call(inputs, run_manager=run_manager)
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/llm.py", line 126, in _call
response = self.generate([inputs], run_manager=run_manager)
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/llm.py", line 138, in generate
return self.llm.generate_prompt(
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 560, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 421, in generate
raise e
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 411, in generate
self._generate_with_cache(
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 632, in _generate_with_cache
result = self._generate(
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_openai/chat_models/base.py", line 522, in _generate
response = self.client.create(messages=message_dicts, **params)
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/_utils/_utils.py", line 277, in wrapper
return func(*args, **kwargs)
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 590, in create
return self._post(
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/_base_client.py", line 1240, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/_base_client.py", line 921, in request
return self._request(
File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/_base_client.py", line 1020, in _request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'detail': 'Invalid max_tokens value, the valid range of max_tokens is [0, 4096]'}

Originally created by @audreyeternal on GitHub (May 16, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/34 I followed the instruction in README about how to utilize deepseek in langchain: ```python model = ChatOpenAI( model="deepseek-chat", openai_api_key=API_KEY, openai_api_base='https://api.deepseek.com/v1', temperature=0.85, max_tokens=8000) ``` However, seems like the `max_tokens` is still restricted to 4k and an error will be raised when intending to integrate the model into the chain and invoke it: ```python qa_with_sources = RetrievalQAWithSourcesChain.from_chain_type( llm=model, chain_type="stuff", retriever=pinecone.as_retriever() ) query = "foobar" result = qa_with_sources.invoke(query) ``` > [!WARNING] > File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/base.py", line 153, in invoke self._call(inputs, run_manager=run_manager) File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 137, in _call output, extra_return_dict = self.combine_docs( File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 244, in combine_docs return self.llm_chain.predict(callbacks=callbacks, **inputs), {} File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/llm.py", line 316, in predict return self(kwargs, callbacks=callbacks)[self.output_key] File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 148, in warning_emitting_wrapper return wrapped(*args, **kwargs) File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/base.py", line 378, in __call__ return self.invoke( File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/base.py", line 163, in invoke raise e File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/base.py", line 153, in invoke self._call(inputs, run_manager=run_manager) File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/llm.py", line 126, in _call response = self.generate([inputs], run_manager=run_manager) File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain/chains/llm.py", line 138, in generate return self.llm.generate_prompt( File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 560, in generate_prompt return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs) File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 421, in generate raise e File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 411, in generate self._generate_with_cache( File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 632, in _generate_with_cache result = self._generate( File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/langchain_openai/chat_models/base.py", line 522, in _generate response = self.client.create(messages=message_dicts, **params) File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/_utils/_utils.py", line 277, in wrapper return func(*args, **kwargs) File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 590, in create return self._post( File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/_base_client.py", line 1240, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/_base_client.py", line 921, in request return self._request( File "/Users/zhouyu/miniconda3/envs/ob_chatbot/lib/python3.10/site-packages/openai/_base_client.py", line 1020, in _request raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: Error code: 400 - {'detail': 'Invalid max_tokens value, the valid range of max_tokens is [0, 4096]'}
bsakaguchi commented 2026-10-26 21:07:00 +00:00
Author
Owner
Copy link

@luofuli commented on GitHub (May 27, 2024):

@audreyeternal The DeepSeek-V2 API currently only supports 32K context length (input + output) with a maximum of 4K tokens for the output.

PS: Although the DeepSeek-V2 model supports 128K context length, we have restricted the API to 32K to ensure efficiency in our services.

<!-- gh-comment-id:2133226978 --> @luofuli commented on GitHub (May 27, 2024): @audreyeternal The DeepSeek-V2 API currently only supports 32K context length (input + output) with a maximum of 4K tokens for the output. PS: Although the DeepSeek-V2 model supports 128K context length, we have restricted the API to 32K to ensure efficiency in our services.
bsakaguchi commented 2026-10-26 21:07:01 +00:00
Author
Owner
Copy link

@Swephoenix commented on GitHub (Jul 30, 2024):

@audreyeternal The DeepSeek-V2 API currently only supports 32K context length (input + output) with a maximum of 4K tokens for the output.

PS: Although the DeepSeek-V2 model supports 128K context length, we have restricted the API to 32K to ensure efficiency in our services.

Like this?

import requests
import json

url = "https://api.deepseek.com/beta/chat/completions"

headers = {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Authorization': 'Bearer [hidden for this chat]'
}

def get_response(messages):
payload = json.dumps({
"messages": messages,
"model": "deepseek-coder",
"frequency_penalty": 0,
"max_tokens": 8192,
"presence_penalty": 0,
"response_format": {"type": "text"},
"stop": None,
"stream": True,
"include_usage": True,
"temperature": 0.7,
"top_p": 1,
"tools": None,
"tool_choice": "none",
"logprobs": False,
"top_logprobs": None
})

response = requests.post(url, headers=headers, data=payload, stream=True)

content = ""
try:
    for line in response.iter_lines(decode_unicode=True):
        if line.startswith("data: "):
            data = line[6:]  # Strip "data: " prefix
            if data == "[DONE]":
                break
            chunk = json.loads(data)
            if 'choices' in chunk:
                chunk_message = chunk['choices'][0]['delta'].get('content', '')
                content += chunk_message
                print(chunk_message, end="", flush=True)
            elif 'usage' in chunk:
                print("\nUsage statistics: ", chunk['usage'])
except json.JSONDecodeError as e:
    print(f"Failed to parse response as JSON. Error: {e}")
    print(f"Response content: {line}")
    return ""

return content.... etc.
<!-- gh-comment-id:2257978361 --> @Swephoenix commented on GitHub (Jul 30, 2024): > @audreyeternal The DeepSeek-V2 API currently only supports 32K context length (input + output) with a maximum of 4K tokens for the output. > > PS: Although the DeepSeek-V2 model supports 128K context length, we have restricted the API to 32K to ensure efficiency in our services. Like this? import requests import json url = "https://api.deepseek.com/beta/chat/completions" headers = { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Authorization': 'Bearer [hidden for this chat]' } def get_response(messages): payload = json.dumps({ "messages": messages, "model": "deepseek-coder", "frequency_penalty": 0, "max_tokens": 8192, "presence_penalty": 0, "response_format": {"type": "text"}, "stop": None, "stream": True, "include_usage": True, "temperature": 0.7, "top_p": 1, "tools": None, "tool_choice": "none", "logprobs": False, "top_logprobs": None }) response = requests.post(url, headers=headers, data=payload, stream=True) content = "" try: for line in response.iter_lines(decode_unicode=True): if line.startswith("data: "): data = line[6:] # Strip "data: " prefix if data == "[DONE]": break chunk = json.loads(data) if 'choices' in chunk: chunk_message = chunk['choices'][0]['delta'].get('content', '') content += chunk_message print(chunk_message, end="", flush=True) elif 'usage' in chunk: print("\nUsage statistics: ", chunk['usage']) except json.JSONDecodeError as e: print(f"Failed to parse response as JSON. Error: {e}") print(f"Response content: {line}") return "" return content.... etc.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-V2#32
No description provided.