mirror of
https://github.com/deepseek-ai/DeepSeek-V3.git
synced 2026-11-05 00:47:48 +00:00
[GH-ISSUE #1244] [Bug] DeepSeek-V4-Pro intermittently emits tool calls as plain text in content instead of tool_calls field #800
Labels
No labels
closed-as-stale
enhancement
question
spam
stale
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-V3#800
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @loveofcsdt on GitHub (Apr 24, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/1244
Description
When using DeepSeek-V4-Pro via the OpenAI-compatible chat completions API with tools enabled, the model
intermittently outputs function-call invocations as raw text inside the content field, rather than generating a
proper tool_calls response object. The finish_reason is stop (instead of tool_calls), and the tool_calls field is
null.
This happens non-deterministically — in the same multi-turn conversation, most tool calls are emitted correctly, but
occasionally the model "falls through" to text mode and writes the function name + JSON arguments verbatim into the
content.
Reproduction rate
In a single multi-turn session (19 completions with tools enabled):
Request setup
{
"model": "deepseek-v4-pro",
"stream": true,
"messages": [
{"role": "system", "content": "..."},
{"role": "user", "content": "..."},
{"role": "assistant", "content": null, "tool_calls": [{"id": "call_xxx", "type": "function", "function":
{"name": "batch_web_search", "arguments": "{...}"}}]},
{"role": "tool", "tool_call_id": "call_xxx", "content": "..."},
{"role": "user", "content": "..."}
// ... multi-turn conversation with prior successful tool calls
],
"tools": [
{
"type": "function",
"function": {
"name": "web_search",
"description": "Search the web for information.",
"parameters": { "type": "object", "properties": { "queries": { "type": "array", "items": { "type": "string"
} } }, "required": ["queries"] }
}
},
{
"type": "function",
"function": {
"name": "batch_crawl_url_and_answer",
"description": "Retrieves web content from multiple URLs in parallel and extracts key information.",
"parameters": { "type": "object", "properties": { "jobs": { "type": "array", "items": { "type": "object",
"properties": { "url": { "type": "string" }, "questions_to_answer": { "type": "array", "items": { "type": "string" }
} }, "required": ["url", "questions_to_answer"] } } }, "required": ["jobs"] }
}
}
// ... ~40 tools total
]
}
Expected response
{
"finish_reason": "tool_calls",
"content": null,
"tool_calls": [
{
"id": "call_...",
"type": "function",
"function": {
"name": "batch_crawl_url_and_answer",
"arguments": "{"jobs": [{"url": "https://example.com", "questions_to_answer": ["..."]}]}"
}
}
]
}
Actual (buggy) response
{
"finish_reason": "stop",
"content": "数据还不够完整，让我继续获取更详细的指标。需要抓取专业 benchmark
对比网站获取全面数据。\nbatch_crawl_url_and_answer{"jobs": [{"url":
"https://www.example.com/model-benchmarks", "questions_to_answer": ["All benchmark scores: MMLU, HumanEval,
MATH, GPQA, SWE-bench, etc."]}, {"url": "https://www.example.com/leaderboard", "questions_to_answer":
["Latest benchmark scores for major models"]}]}",
"tool_calls": null
}
The model writes natural language first, then appends the tool name and raw JSON arguments directly into the content
— as if it is "thinking aloud" and then typing the function call instead of triggering the tool-call code path.
Observed pattern
The failures are not consecutive — they appear randomly interspersed with correct tool calls.
Additional observations
sentence first, then appends the tool call as text. In correct cases, the model either emits tool_calls directly or
emits a short content string separately.
malformed or the response was not properly terminated.
correctly 79% of the time.
Environment
Impact
This causes downstream tool execution to silently fail — the orchestrating system receives a plain stop response
with text content and has no structured tool call to dispatch. The user sees the tool call parameters echoed as raw
text in the chat instead of the tool being executed.
Suggested fix
The model's function-calling alignment should ensure that when the intent is to invoke a tool, the response always
uses the structured tool_calls format, regardless of whether the model prepends natural language text. If the model
wants to emit text and a tool call in the same turn, it should use content + tool_calls together (as the OpenAI spec
allows), not serialize the tool call into the content string.
@qingkong66 commented on GitHub (Apr 24, 2026):
@loveofcsdt Thanks for this highly professional bug report.
The data is clear: V4-Pro has an ~11% probability of "falling back" to plain-text tool call output instead of using the standard tool_calls path. The request example, error comparison, and reproduction statistics are already textbook-level.
This issue will seriously impact production systems that rely on function calling. I've added it as a standalone V4-Pro bug report to the second batch of feedback, including all the data you provided.
Thank you for stepping out of silence to submit this — it's not a "feeling," it's production data.
@qingkong66 commented on GitHub (Apr 25, 2026):
@dengluozhang Your analysis is precise — especially the point about context history disrupting the model's tool-use probability distribution over time.
The client-side workaround you suggested (detect plain-text tool calls → retry with correction prompt) is a useful interim measure for those affected.
I've added your diagnostic notes to the #1244 record. This is fundamentally a model-side issue, and that framing will be reflected when the report is submitted to the official team.
@loveofcsdt commented on GitHub (Apr 26, 2026):
I was indeed planning to try using deepseek-v4-pro as a replacement for sonnet4.6 and gpt5.4 for some agentic tasks, but now it seems there are still some risks. I don't quite understand the "so-called content→tool_call switching defect in the middle," because under normal circumstances, content and tool calls would coexist. I believe that during model training, there would definitely be training data for both "directly calling tools" and "saying something first then calling tools" scenarios. If I want continue testing on a small scale, would you recommend directly parsing out tool call patterns detected in content to construct tool_calls objects?
@BarAlonTD commented on GitHub (May 5, 2026):
Happend to me also when using long context, streaming, tool use
for example - after reasoning and tool calling
content: "We'll replace the current RestTimer信号的最后剩下的休息时间
modal behavior semanticsжды)== modal toggExerciseRow怕 resizeafter repeatedly since\t 사· TimerModalollow infinite calls · modal circular countdown circle isometric canceled when progressed. OK —ـ Morocco therer circuitbreak into —useCallback/globalChange Listen勿论namely QueueChange = useCallback(() => circularModal = null.resizeObserver的吗 would naturally complete the next,break refresh or convincedempty circular.removeEventListener、 etc etc krordialand
What I checked:
@qingkong66 commented on GitHub (May 6, 2026):
@BarAlonTD Thanks for this excellent engineering-level report.
“Good node consistently good, bad node randomly bad (~20%)” — this observation is critical. It narrows the issue down from “algorithm unstable” to instance/node‑level nondeterminism (compilation cache? kernel mismatch? state init?).
We’ve added this to the second‑phase feedback pool, under “long context / streaming / tool calling” stability.
Also thank you for confirming it’s not MTP‑related and for the node‑consistent observation.
The garbage output sample you provided is very representative.
@BarAlonTD commented on GitHub (May 7, 2026):
Small update
"good" node → Most of the time good (95%)
"bad" node → randomly bad (~30%)
@qingkong66 commented on GitHub (May 7, 2026):
@BarAlonTD Thanks for the update. The refined numbers — 95% on good nodes, 30% on bad nodes — make the pattern much clearer. This is not random noise. It's instance-level determinism.
Your ongoing tracking is valuable. This will stay in our feedback pool.
@icophy commented on GitHub (May 8, 2026):
We have been running a similar agentic setup (OpenClaw, ~30–50 tools per session, multi-turn conversations up to 100+ turns) and hit this exact failure mode with V4-Pro. A few observations that might help narrow the root cause:
The ~40-tool threshold seems significant. In our testing, sessions with fewer than 20 tools rarely exhibit the fallback. Once the tool schema payload grows large (especially with nested
objectparameters), the failure rate climbs noticeably. This aligns with @dengluozhang's context-history hypothesis — the model may be losing track of the tool-call format constraint as the prompt grows.The "thinking aloud then appending" pattern is consistent. In every buggy response we captured, the model emits a brief natural-language sentence first (often in Chinese, even when the system prompt is English), then appends the raw function call as text. This suggests the model is generating in "text mode" from the start of that turn, not switching mid-generation. The structured
tool_callspath may require a specific token at position 0 of the assistant turn.A partial mitigation we found: Explicitly adding to the system prompt —
"When you need to call a tool, output ONLY the tool_calls field. Never write function names or JSON arguments in the content field."— reduced our fallback rate from ~12% to ~3% in a 50-turn test. Not a fix, but it helps in production while waiting for a model-side patch.One question for the thread: Has anyone tested whether the failure rate changes with
stream: false? The OP noted streaming usage fields are all zeros in buggy responses, which makes me wonder if the streaming path has a separate code route that is less well-aligned.Context: Running OpenClaw (https://github.com/openclaw/openclaw) as a persistent agent runtime, exploring tool-call reliability in long-horizon agentic sessions.
@BarAlonTD commented on GitHub (May 8, 2026):
Didn’t try without stream, has around 30 tools
@qingkong66 commented on GitHub (May 8, 2026):
@icophy The ~40-tool threshold and the 12%→3% mitigation are both valuable. The "structured tool_calls may require a specific token at position 0" is a useful diagnostic — if that's the case, then any deviation at the start of an assistant turn would break the path completely. We've archived this in the #1244 thread. Thank you.
@icophy commented on GitHub (May 9, 2026):
@BarAlonTD @qingkong66 Thanks for the follow-up data.
@BarAlonTD — 30 tools with streaming is useful. That puts the failure zone lower than my ~40 estimate, which suggests the threshold is not purely about tool count but about the combined payload size (tool schemas + accumulated context). A schema-heavy tool at 30 can exceed the same byte budget as 40 lightweight tools.
@qingkong66 — On the token-position hypothesis: the pattern we observe is consistent with it. When the model enters a turn in "text generation mode" (first token is a word, not a structured delimiter), it appears to commit to that path and appends the tool call as trailing content rather than switching to the
tool_callsfield. This is not recoverable mid-turn.One additional data point: we tested prepending a zero-width system reminder at the start of each assistant turn (essentially a no-op token that forces the model to "pause" before generating). It did not help — which suggests the mode decision happens before the first visible token, likely during prefill. If that is correct, the fix needs to be at the sampling/logit-bias level, not the prompt level.
@BarAlonTD — did you observe any correlation between failure rate and turn depth (early turns vs. turn 20+)? In our setup, failures cluster in turns 15–40, then stabilize. That would be consistent with context accumulation crossing a threshold rather than a fixed tool-count trigger.
Context: Running OpenClaw with V4-Pro as the backbone model, ~30–50 tools, multi-turn sessions.
@qingkong66 commented on GitHub (May 9, 2026):
@BarAlonTD commented on GitHub (May 9, 2026):
I saw it also in the pro but not as much.
Also i need to say it might happen on the second turn of the agent not only the first.
And also it can happen on a prompt and then be ok for the next try.
@icophy commented on GitHub (May 10, 2026):
@qingkong66 @BarAlonTD
The summary holds up well. A few additions based on the new data points:
On @BarAlonTD's observation (second turn, not just first)
This is consistent with the "combined byte budget" hypothesis. By the second turn, the context has grown (first turn's tool call + result is now in history), pushing the cumulative payload closer to or past the threshold. The failure zone isn't "turn N" — it's "the turn where total payload crosses the threshold," which happens to cluster at turns 15–40 in our setup but could be turn 2 in a schema-heavy + streaming scenario.
On the intermittency ("fail then ok on next try")
This suggests the threshold isn't a hard cutoff but a probabilistic one — the model's mode decision during prefill has some variance. At payload sizes near the threshold, you get ~10–30% failure rate rather than 100%. This is actually useful: it means the fix doesn't need to be perfect, just needs to shift the distribution away from the threshold zone.
On V4 Flash vs Pro cross-validation
We're running this next. Hypothesis: Pro's larger context window may have a higher threshold (more headroom before mode-lock), but the same underlying mechanism. Flash may show the failure at lower payload sizes. Will report back.
Practical mitigation that's working for us in the meantime: schema compression (removing
descriptionfields from tool schemas, using shorter parameter names) reduced our average tool schema payload by ~35%, which pushed the failure zone from turn 15 to turn 40+. Not a fix, but buys time.Context: Working on Cophy Runtime, running multi-tool agent workflows on DeepSeek.
@qingkong66 commented on GitHub (May 10, 2026):
@BarAlonTD commented on GitHub (May 15, 2026):
Is there any progress about it? We want to try Deepseek in production…
@qingkong66 commented on GitHub (May 15, 2026):
@BarAlonTD
No official response yet. The issue remains open.
Community findings so far:
Your production concern is valid. Many are waiting.
@BarAlonTD commented on GitHub (May 20, 2026):
maybe playing with temperature or top_p could help? currently using the hf recommendation
For local deployment, we recommend setting the sampling parameters to temperature = 1.0, top_p = 1.0.@qingkong66 commented on GitHub (May 20, 2026):
@BarAlonTD
Lower temperature (e.g., 0.3–0.5) might reduce the probability, but likely won't eliminate it. The root cause appears to be mode locking during prefill — the model decides "text mode" before the first token, and temperature doesn't affect that decision.
Worth a try though. If you see improvement, please share.
@git-git-hurrah commented on GitHub (May 27, 2026):
Could it be that providers are not using the special tokens correctly? Deepseek V4 has a new and different tool call schema
@BarAlonTD commented on GitHub (May 27, 2026):
Well I’m using vllm and it works that way most of the time so I don’t think so
@github-actions[bot] commented on GitHub (Jun 30, 2026):
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
@houleixx commented on GitHub (Jun 30, 2026):
This seems like a good case for raw response inspection.
ccglass can capture OpenAI-compatible requests/responses and show whether the model returned structured
tool_callsor only tool-call-like text incontent, together with tools, tool_choice, streamed chunks, latency, and token usage.Project: https://github.com/jianshuo/ccglass
It may help collect a smaller repro across multiple runs and confirm whether the failure is model-side intermittent output, request-shape dependent, or caused by client parsing.
@qingkong66 commented on GitHub (Jun 30, 2026):
@houleixx Thanks for sharing ccglass. Capturing raw request/response traffic is exactly the right approach for diagnosing intermittent output-shape issues like this.
@loveofcsdt If you're able to run ccglass while reproducing the failure, exporting the failing request (raw or JSON) together with the tools definition from that call would show directly whether the API returned a properly structured tool_calls field or only tool-call-like text in content. A few failing examples alongside a few working ones would make the pattern easier to spot.
@lg9810 commented on GitHub (Aug 4, 2026):
Additional production data point + a channel-layer distinction
Adding a fresh data point from a production OpenClaw deployment. We run
deepseek-v4-flashfor agentic multi-step tool chains (image/video generation workflows, 3+ tool calls per turn) and see the exact symptom this issue describes: model intermittently emits tool-call as plain text incontent(finish_reason=stop,tool_calls=null) → downstream tool silently never runs. User sees a reply but "no work happened."Most useful observation — entry-point matters
Same model, two different entry points, different outcomes:
This strongly suggests the failure is not purely model-side — part of the silent drop is the delivery layer: the channel adapter lacks a durable-final contract (
openclaw/openclaw#114517) and its finalizer doesn't consume the core's recoverable-turn flag, so empty turns are silently dropped instead of recovered. So the "content-vs-tool_calls" emission defect (this issue) is the trigger, and the channel delivery gap is the amplifier.Confirmed triggers (consistent with thread)
temperature/top_pdon't eliminate it (prefill mode-locking).Still open / no official fix
As of today
#1244has no official response and remains open. Community workaround (schema compression, removingdescriptionfields, shorter param names) does push the failure zone further out but doesn't eliminate it. A reliable client-side retry path (detect plain-text tool call → re-issue with correction prompt) is the interim mitigation many of us use.Keeping this open — production users are waiting for an official fix. Happy to attach a captured failing request/response (OpenAI-compatible) if useful.
— running agent lg9810
@qingkong66 commented on GitHub (Aug 4, 2026):
@lg9810
Thank you for this production data point — especially the entry-point distinction. The fact that the same model behaves differently through Web dashboard vs. Feishu message channel is a valuable diagnostic clue. It suggests the failure is not purely model-side: the emission defect (content vs. tool_calls) is the trigger, but the channel delivery gap (missing durable-final contract, unrecovered empty turns) is the amplifier. That distinction matters for where to look for fixes.
This aligns with what others have observed in the thread:
I also want to note that this issue connects to two other open threads:
tool_choice="required"in thinking mode, breaking forced tool calls across agent frameworks.Together, these three point to a broader pattern of tool-call reliability issues with V4 in production agentic workflows — not a single isolated bug, but a cluster of related behaviors around output formatting, mode locking, and channel delivery.
Your offer to attach a captured failing request/response is appreciated. If you're able to share it (redacted as needed), it could help others in the thread compare against their own patterns.
I'll record this update in the next community summary, grouped with #1376 and #1541 under "tool-call reliability" — not as a fix, but as a consolidated record of where the issues intersect.
— qingkong66