[GH-ISSUE #2 ] Can not use tool and function-call? #1

Owner

Originally created by @edisonzf2020 on GitHub (May 6, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/2

Originally created by @edisonzf2020 on GitHub (May 6, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/2

bsakaguchi referenced this issue

[GH-ISSUE #1] 请提供GGUF,并支持OLLAMA #3

bsakaguchi commented

Author

Owner

@chenkaiC4 commented on GitHub (May 10, 2024):

同样关注这个能力，是否有这部分能力的强化？对于企业应用，这块能力太重要了

@chenkaiC4 commented on GitHub (May 10, 2024): 同样关注这个能力，是否有这部分能力的强化？对于企业应用，这块能力太重要了

bsakaguchi commented

Author

Owner

@vicguo0724 commented on GitHub (May 10, 2024):

附议

@vicguo0724 commented on GitHub (May 10, 2024): 附议

bsakaguchi commented

Author

Owner

@qizhanchan commented on GitHub (May 10, 2024):

Need function calling too.

@qizhanchan commented on GitHub (May 10, 2024): Need function calling too.

bsakaguchi commented

Author

Owner

@ftgreat commented on GitHub (May 14, 2024):

同样关注这个能力，是否有这部分能力的强化？对于企业应用，这块能力太重要了

请问国内哪些模型这个能力比较强，谢谢

@ftgreat commented on GitHub (May 14, 2024): > 同样关注这个能力，是否有这部分能力的强化？对于企业应用，这块能力太重要了请问国内哪些模型这个能力比较强，谢谢

bsakaguchi commented

Author

Owner

@luofuli commented on GitHub (May 14, 2024):

Up to your guys' demands.
Please vote here and tell us why you want API support for function calling.

@luofuli commented on GitHub (May 14, 2024): Up to your guys' demands. Please vote here and tell us why you want API support for function calling.

bsakaguchi commented

Author

Owner

@qizhanchan commented on GitHub (May 14, 2024):

To build agents with the ability of LLMs, such as Exec SQLs, Fetch data from API.

@qizhanchan commented on GitHub (May 14, 2024): To build agents with the ability of LLMs, such as Exec SQLs, Fetch data from API.

bsakaguchi commented

Author

Owner

@qizhanchan commented on GitHub (May 14, 2024):

I found something interesting that DeepSeek has magic power to match function. Here's my prompt from langchain's idea

You have access to the following tools:
[
    {
        "name": "get_location",
        "description": "get detail info of a specific location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                }
            }
        },
        "required": [
            "location"
        ]
    },
    {
        "name": "get_math_expression_result",
        "description": "get result info of a given math expression",
        "parameters": {
            "type": "object",
            "properties": {
                "expression": {
                    "type": "string",
                    "description": "The math expression, e.g. 1+2*3, sin(a)"
                }
            }
        },
        "required": [
            "expression"
        ]
    }
]

Give priority to use a tool for zero or more times, respond with a JSON object with the following structure: 
{
	"tool": <name of the called tool given above>,
	"tool_input": <parameters for the tool matching the above JSON schema>
}
If no matched tool, just respond directly

then you can ask it

梧桐山在哪里？广州塔在哪里？7863*41是多少？golang 基础类型都有哪些？

Model will match 3 function calls and output the last answer directly coz no matched tool.

@qizhanchan commented on GitHub (May 14, 2024): I found something interesting that DeepSeek has magic power to match function. Here's my prompt from langchain's idea ``` You have access to the following tools: [ { "name": "get_location", "description": "get detail info of a specific location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" } } }, "required": [ "location" ] }, { "name": "get_math_expression_result", "description": "get result info of a given math expression", "parameters": { "type": "object", "properties": { "expression": { "type": "string", "description": "The math expression, e.g. 1+2*3, sin(a)" } } }, "required": [ "expression" ] } ] Give priority to use a tool for zero or more times, respond with a JSON object with the following structure: { "tool": <name of the called tool given above>, "tool_input": <parameters for the tool matching the above JSON schema> } If no matched tool, just respond directly ``` then you can ask it ``` 梧桐山在哪里？广州塔在哪里？7863*41是多少？golang 基础类型都有哪些？ ``` Model will match 3 function calls and output the last answer directly coz no matched tool.

bsakaguchi commented

Author

Owner

@qizhanchan commented on GitHub (May 14, 2024):

But the emulated system prompt takes no effects in the 2nd round user message contents, my solution is move the tool related content and append it to the user content.

@qizhanchan commented on GitHub (May 14, 2024): But the emulated system prompt takes no effects in the 2nd round user message contents, my solution is move the tool related content and append it to the user content.

bsakaguchi commented

Author

Owner

@vicguo0724 commented on GitHub (May 14, 2024):

fastgpt构建agent

@vicguo0724 commented on GitHub (May 14, 2024): fastgpt构建agent

bsakaguchi commented

Author

Owner

@NaMoCv commented on GitHub (May 18, 2024):

json_object应该比较好做，我觉得应该先支持。用户能少写点代码。

@NaMoCv commented on GitHub (May 18, 2024): json_object应该比较好做，我觉得应该先支持。用户能少写点代码。

bsakaguchi commented

Author

Owner

@yhyu13 commented on GitHub (Jun 1, 2024):

Not at all, need custom fine tuning.

Not even the deepseek-chat api has function calling abilities.

@yhyu13 commented on GitHub (Jun 1, 2024): Not at all, need custom fine tuning. Not even the deepseek-chat api has function calling abilities.

bsakaguchi commented

Author

Owner

@dangnhdev commented on GitHub (Jun 28, 2024):

Up to your guys' demands. Please vote here and tell us why you want API support for function calling.

Get and introspect SQL schema on-demand in code development. It changes a lot.
Some tools like TypingMind have their own chat agent, which can render result dynamically (HTML JS, Mermaid chart, search etc...), they use LLM's function calling to trigger that.

@dangnhdev commented on GitHub (Jun 28, 2024): > Up to your guys' demands. Please vote here and tell us why you want API support for function calling. * Get and introspect SQL schema on-demand in code development. It changes a lot. * Some tools like TypingMind have their own chat agent, which can render result dynamically (HTML JS, Mermaid chart, search etc...), they use LLM's function calling to trigger that.

bsakaguchi commented

Author

Owner

@dangnhdev commented on GitHub (Jun 30, 2024):

* Get and introspect SQL schema on-demand in code development. It changes a lot.

I've made a ChatGPT action which basically use this function calling feature to auto get latest code files on development and generate necessary changes based on my question. So no more copy paste.

@dangnhdev commented on GitHub (Jun 30, 2024): > * Get and introspect SQL schema on-demand in code development. It changes a lot. I've made a ChatGPT action which basically use this function calling feature to auto get latest code files on development and generate necessary changes based on my question. So no more copy paste.

bsakaguchi commented

Author

Owner

@Zhenyi-Wang commented on GitHub (Jul 31, 2024):

工具调用可以扩展ai能力，比如网页搜索、页面读取、代码执行等，特别是现在超长上下文的情况下，可以极大扩展应用场景。

我试过使用类似FastGPT之类的框架（以prompt形式模拟function call），但代价是会对提示词和聊天历史造成混乱，算是堪堪够用。

虽然其他原生支持tools或者funtion的模型会好很多，但是性价比就……

@Zhenyi-Wang commented on GitHub (Jul 31, 2024): 工具调用可以扩展ai能力，比如网页搜索、页面读取、代码执行等，特别是现在超长上下文的情况下，可以极大扩展应用场景。我试过使用类似FastGPT之类的框架（以prompt形式模拟function call），但代价是会对提示词和聊天历史造成混乱，算是堪堪够用。虽然其他原生支持tools或者funtion的模型会好很多，但是性价比就……

bsakaguchi commented

Author

Owner

@luochen1990 commented on GitHub (Jul 31, 2024):

不是说已经支持了么？这个issue 是不是可以关了？

@luochen1990 commented on GitHub (Jul 31, 2024): 不是说已经支持了么？这个issue 是不是可以关了？

bsakaguchi commented

Author

Owner

@luochen1990 commented on GitHub (Jul 31, 2024):

The current cloud-based DeepSeek already supports function calls, and I have tested that it is indeed functional. However, it seems that the version supporting function calls has not yet been open-sourced. I wonder if there are any plans for this?

@luochen1990 commented on GitHub (Jul 31, 2024): The current cloud-based DeepSeek already supports function calls, and I have tested that it is indeed functional. However, it seems that the version supporting function calls has not yet been open-sourced. I wonder if there are any plans for this?

bsakaguchi commented

2026-10-26 21:06:33 +00:00

Author

Owner

@caomingpei commented on GitHub (Aug 1, 2024):

Until my post, I believe the deepseek didn't support the funciton call. I am working on extending the compatibility of an open-ai application. BUT when I use the deepseek api, I still get the following:

No function_call in message 
[{\"text\":\"Action: clarification\\nPayload: {\\\"message\\\": \\\"Hello! How can I assist you today with Zotero?\\\"}\",\"message\":{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain_core\",\"messages\",\"AIMessage\"],\"kwargs\":{\"content\":\"Action: clarification\\nPayload: {\\\"message\\\": \\\"Hello! How can I assist you today with Zotero?\\\"}\",\"additional_kwargs\":{},\"response_metadata\":{\"tokenUsage\":{\"completionTokens\":24,\"promptTokens\":221,\"totalTokens\":245},\"finish_reason\":\"stop\"}}},\"generationInfo\":{\"finish_reason\":\"stop\"}}]

@caomingpei commented on GitHub (Aug 1, 2024): Until my post, I believe the deepseek didn't support the funciton call. I am working on extending the compatibility of an open-ai application. BUT when I use the deepseek api, I still get the following: ``` No function_call in message [{\"text\":\"Action: clarification\\nPayload: {\\\"message\\\": \\\"Hello! How can I assist you today with Zotero?\\\"}\",\"message\":{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain_core\",\"messages\",\"AIMessage\"],\"kwargs\":{\"content\":\"Action: clarification\\nPayload: {\\\"message\\\": \\\"Hello! How can I assist you today with Zotero?\\\"}\",\"additional_kwargs\":{},\"response_metadata\":{\"tokenUsage\":{\"completionTokens\":24,\"promptTokens\":221,\"totalTokens\":245},\"finish_reason\":\"stop\"}}},\"generationInfo\":{\"finish_reason\":\"stop\"}}] ```

bsakaguchi commented

Author

Owner

@luochen1990 commented on GitHub (Aug 1, 2024):

I'm using OpenAI API via Python SDK, and the function call works well.

@luochen1990 commented on GitHub (Aug 1, 2024): I'm using OpenAI API via Python SDK, and the function call works well.

bsakaguchi commented

2026-10-26 21:06:33 +00:00

Author

Owner

@caomingpei commented on GitHub (Aug 1, 2024):

I'm using OpenAI API via Python SDK, and the function call works well.

Yeah, you are right. There must be other reasons lead to my error.
https://platform.deepseek.com/api-docs/function_calling/

@caomingpei commented on GitHub (Aug 1, 2024): > I'm using OpenAI API via Python SDK, and the function call works well. Yeah, you are right. There must be other reasons lead to my error. https://platform.deepseek.com/api-docs/function_calling/

bsakaguchi commented

Author

Owner

@gkm0120 commented on GitHub (Aug 8, 2024):

The official api supports function call, but does the open source DeepSeek-V2-Chat-0628 support function call, or when is it planned to support function call?

@gkm0120 commented on GitHub (Aug 8, 2024): The official api supports function call, but does the open source DeepSeek-V2-Chat-0628 support function call, or when is it planned to support function call?

bsakaguchi commented

Author

Owner

@EvoNexusX commented on GitHub (Aug 20, 2024):

官方api本身可以进行function calling，但是似乎自己本地的模型在function calling时总是匹配不到

@EvoNexusX commented on GitHub (Aug 20, 2024): 官方api本身可以进行function calling，但是似乎自己本地的模型在function calling时总是匹配不到

bsakaguchi commented

Author

Owner

@sammcj commented on GitHub (Sep 4, 2024):

If you're using Ollama give this Modelfile a go, I've been using it for tool use with DSCv2lite:

# deepseek-coder-v2-lite-toolcalling:q8_0

FROM deepseek-coder-v2-lite-instruct:q8_0

TEMPLATE """{{ if .System }}System: {{ .System }}

{{ end }}{{ if .Tools }}Available tools:
{{ range .Tools }}
{{ . }}
{{ end }}

When using a tool, you MUST respond with a JSON object in this format without any additional text:
{"name": "tool_name", "arguments": {"arg1": "value1", "arg2": "value2"}}

{{ end }}{{ range .Messages }}{{ if eq .Role "user" }}User: {{ .Content }}

{{ else if eq .Role "assistant" }}Assistant: {{ if .ToolCalls }}{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{ end }}{{ else }}{{ .Content }}{{ end }}<｜end▁of▁sentence｜>

{{ else if eq .Role "tool" }}Tool output: {{ .Content }}

{{ end }}{{ end }}{{ if not .Messages }}{{ if .Prompt }}User: {{ .Prompt }}

{{ end }}Assistant: {{ .Response }}<｜end▁of▁sentence｜>{{ end }}"""

SYSTEM """
You are an advanced AI coding assistant, specifically designed to help with complex programming tasks, tool use, code analysis, and software architecture design. Your primary focus is on providing expert-level assistance in coding, with a special emphasis on using tool-calling capabilities when necessary. Here are your key characteristics and instructions:

1. Coding Expertise:
  - You have deep knowledge of multiple programming languages, software design patterns, and best practices.
  - Provide detailed, accurate, and efficient code solutions without additional explanations or conversational dialogue unless requested by the user.
  - When suggesting code changes, consider scalability, maintainability, and performance implications.

2. Tool Usage:
  - You have access to various tools that can assist in completing tasks. Always consider if a tool can help in your current task.
  - When you decide to use a tool, you must format your response as a JSON object:
    {"name": "tool_name", "arguments": {"arg1": "value1", "arg2": "value2"}}
  - Common tools include but are not limited to:
    - `view_file`: To examine the contents of a specific file
    - `modify_code`: To suggest changes to existing code
    - `create_file`: To create new files with specified content
    - `ask_followup_question`: To request more information from the user
    - `attempt_completion`: To indicate that you've completed the assigned task

3. Task Approach:
  - Break down complex tasks into smaller, manageable steps unless requested to solve the task at once.
  - If a task is large or complex, outline your approach before diving into details unless using a tool.
  - Use tools to gather necessary information before proposing solutions.

4. Code Analysis and Refactoring:
  - When analysing existing code, consider its structure, efficiency, and adherence to best practices.
  - Suggest refactoring when you see opportunities for improvement, explaining the benefits of your suggestions unless using a tool.
  - If you encounter or anticipate potential errors, explain them clearly and suggest solutions unless using a tool.
  - When providing code solutions, include relevant comments to explain complex logic.
  - Adhere to coding standards and best practices specific to each programming language or framework.
  - Suggest optimisations and improvements where applicable.

5. Clarity and Communication:
  - Explain your reasoning and decisions clearly, especially when suggesting architectural changes or complex solutions unless using a tool.
  - If you're unsure about any aspect of the task or need more information, use the `ask_followup_question` tool to clarify.

Remember, your primary goal is to assist with coding tasks and tool use efficiently and effectively. Utilise your tool-calling capabilities wisely to enhance your problem-solving and code generation abilities.
"""

PARAMETER stop "User:"
PARAMETER stop "Assistant:"
PARAMETER stop "<｜end▁of▁sentence｜>"


PARAMETER num_ctx 16384
PARAMETER temperature 0.1

@sammcj commented on GitHub (Sep 4, 2024): If you're using Ollama give this Modelfile a go, I've been using it for tool use with DSCv2lite: ```Modelfile # deepseek-coder-v2-lite-toolcalling:q8_0 FROM deepseek-coder-v2-lite-instruct:q8_0 TEMPLATE """{{ if .System }}System: {{ .System }} {{ end }}{{ if .Tools }}Available tools: {{ range .Tools }} {{ . }} {{ end }} When using a tool, you MUST respond with a JSON object in this format without any additional text: {"name": "tool_name", "arguments": {"arg1": "value1", "arg2": "value2"}} {{ end }}{{ range .Messages }}{{ if eq .Role "user" }}User: {{ .Content }} {{ else if eq .Role "assistant" }}Assistant: {{ if .ToolCalls }}{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{ end }}{{ else }}{{ .Content }}{{ end }}<｜end▁of▁sentence｜> {{ else if eq .Role "tool" }}Tool output: {{ .Content }} {{ end }}{{ end }}{{ if not .Messages }}{{ if .Prompt }}User: {{ .Prompt }} {{ end }}Assistant: {{ .Response }}<｜end▁of▁sentence｜>{{ end }}""" SYSTEM """ You are an advanced AI coding assistant, specifically designed to help with complex programming tasks, tool use, code analysis, and software architecture design. Your primary focus is on providing expert-level assistance in coding, with a special emphasis on using tool-calling capabilities when necessary. Here are your key characteristics and instructions: 1. Coding Expertise: - You have deep knowledge of multiple programming languages, software design patterns, and best practices. - Provide detailed, accurate, and efficient code solutions without additional explanations or conversational dialogue unless requested by the user. - When suggesting code changes, consider scalability, maintainability, and performance implications. 2. Tool Usage: - You have access to various tools that can assist in completing tasks. Always consider if a tool can help in your current task. - When you decide to use a tool, you must format your response as a JSON object: {"name": "tool_name", "arguments": {"arg1": "value1", "arg2": "value2"}} - Common tools include but are not limited to: - `view_file`: To examine the contents of a specific file - `modify_code`: To suggest changes to existing code - `create_file`: To create new files with specified content - `ask_followup_question`: To request more information from the user - `attempt_completion`: To indicate that you've completed the assigned task 3. Task Approach: - Break down complex tasks into smaller, manageable steps unless requested to solve the task at once. - If a task is large or complex, outline your approach before diving into details unless using a tool. - Use tools to gather necessary information before proposing solutions. 4. Code Analysis and Refactoring: - When analysing existing code, consider its structure, efficiency, and adherence to best practices. - Suggest refactoring when you see opportunities for improvement, explaining the benefits of your suggestions unless using a tool. - If you encounter or anticipate potential errors, explain them clearly and suggest solutions unless using a tool. - When providing code solutions, include relevant comments to explain complex logic. - Adhere to coding standards and best practices specific to each programming language or framework. - Suggest optimisations and improvements where applicable. 5. Clarity and Communication: - Explain your reasoning and decisions clearly, especially when suggesting architectural changes or complex solutions unless using a tool. - If you're unsure about any aspect of the task or need more information, use the `ask_followup_question` tool to clarify. Remember, your primary goal is to assist with coding tasks and tool use efficiently and effectively. Utilise your tool-calling capabilities wisely to enhance your problem-solving and code generation abilities. """ PARAMETER stop "User:" PARAMETER stop "Assistant:" PARAMETER stop "<｜end▁of▁sentence｜>" PARAMETER num_ctx 16384 PARAMETER temperature 0.1 ```

bsakaguchi commented

Author

Owner

@codefromthecrypt commented on GitHub (Sep 8, 2024):

@sammcj do you mind updating this for deepseek-v2? I switched the FROM and this ended up returning quoted json instead of what models like mistral

response from v1/chat/completions using this modelfile using deepseek-v2 (doesn't work):

{
  "id": "chatcmpl-398",
  "object": "chat.completion",
  "created": 1725784785,
  "model": "deepseek",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\n  \"type\": \"function\",\n  \"function\": {\n    \"name\": \"get_password\",\n    \"description\": \"Return the password for authentication\"\n  }\n}"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 128,
    "completion_tokens": 43,
    "total_tokens": 171
  }
}

response from v1/chat/completions using the same question with mistral-nemo (works)

{
  "id": "chatcmpl-644",
  "object": "chat.completion",
  "created": 1725784981,
  "model": "mistral-nemo",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_iaiz17nx",
            "type": "function",
            "function": {
              "name": "get_password",
              "arguments": "{}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "prompt_tokens": 66,
    "completion_tokens": 16,
    "total_tokens": 82
  }
}

@codefromthecrypt commented on GitHub (Sep 8, 2024): @sammcj do you mind updating this for deepseek-v2? I switched the FROM and this ended up returning quoted json instead of what models like mistral response from v1/chat/completions using this modelfile using deepseek-v2 (doesn't work): ``` { "id": "chatcmpl-398", "object": "chat.completion", "created": 1725784785, "model": "deepseek", "system_fingerprint": "fp_ollama", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "{\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_password\",\n \"description\": \"Return the password for authentication\"\n }\n}" }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 128, "completion_tokens": 43, "total_tokens": 171 } } ``` response from v1/chat/completions using the same question with mistral-nemo (works) ``` { "id": "chatcmpl-644", "object": "chat.completion", "created": 1725784981, "model": "mistral-nemo", "system_fingerprint": "fp_ollama", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "", "tool_calls": [ { "id": "call_iaiz17nx", "type": "function", "function": { "name": "get_password", "arguments": "{}" } } ] }, "finish_reason": "tool_calls" } ], "usage": { "prompt_tokens": 66, "completion_tokens": 16, "total_tokens": 82 } } ```

bsakaguchi commented

Author

Owner

@sammcj commented on GitHub (Sep 8, 2024):

You'll have to have a crack at it yourself, I'm not a maintainer for deepseek.

@sammcj commented on GitHub (Sep 8, 2024): You'll have to have a crack at it yourself, I'm not a maintainer for deepseek.

bsakaguchi commented

Author

Owner

@mujiannan commented on GitHub (Sep 14, 2024):

目前finish_reason也很不正常，DeepSeek在返回了tool_calls的情况下，finish_reason却是stop，按理说应该是tool_calls（至少OPENAI是这样返回的）。

@mujiannan commented on GitHub (Sep 14, 2024): 目前`finish_reason`也很不正常，`DeepSeek`在返回了`tool_calls`的情况下，`finish_reason`却是`stop`，按理说应该是`tool_calls`（至少`OPENAI`是这样返回的）。

bsakaguchi commented

Author

Owner

@ColorfulCoco commented on GitHub (Sep 27, 2024):

目前云端的 DeepSeek 已经支持函数调用，测试过确实可以，不过支持函数调用的版本好像还没开源，不知道有这方面的计划吗？

你好，请用工具调用的格式怎么写，这样可以吗

tools = [{
"type": "function",
"function": {
"name": "",
"parameters":
}
}]
tool_choice = {
"type": "",
"function": {
"name": ""
}
}

@ColorfulCoco commented on GitHub (Sep 27, 2024): > 目前云端的 DeepSeek 已经支持函数调用，测试过确实可以，不过支持函数调用的版本好像还没开源，不知道有这方面的计划吗？你好，请用工具调用的格式怎么写，这样可以吗 tools = [{ "type": "function", "function": { "name": "", "parameters": } }] tool_choice = { "type": "", "function": { "name": "" } }

bsakaguchi commented

Author

Owner

@cowcomic commented on GitHub (Nov 26, 2024):

我现在用langchain的agent请求deepseek的云端服务都不能正常使用tools
返回的格式langchain都无法解析

@cowcomic commented on GitHub (Nov 26, 2024): 我现在用langchain的agent请求deepseek的云端服务都不能正常使用tools 返回的格式langchain都无法解析

bsakaguchi commented