[GH-ISSUE #300] Critical: bug: a cut stream leaves truncated tool-call arguments, json.loads on the accumulator kills the whole chat task and persists an error turn every retry #213

New issue

Open

opened 2026-10-18 14:03:50 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-18 14:03:50 +00:00

Owner

Originally created by @Classic298 on GitHub (Sep 21, 2026).
Original GitHub issue: https://github.com/open-webui/computer/issues/300

Same root cause as #252 (the unguarded json.loads), but this is the sharper end of it: what happens when the accumulator is not just empty but truncated mid-string by a cut stream, and what it does to the chat. Filing separately because the blast radius is different from the "empty/short JSON" case there.

What happened, on cptr 0.9.21 behind an OpenAI-compatible proxy:

The proxy cut the SSE stream mid-argument (broken pipe on its side, logged at the same second as the crash). cptr had accumulated arguments_json by concatenating deltas, and the accumulation ended mid-string: {"content": "\"\"\"Regression tests for ... and then nothing. When a later chunk carried finish_reason: tool_calls, cptr ran json.loads on that truncated string and raised:

json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 13 (char 12)

at json.loads(tc["arguments_json"]) inside stream_openai_completions, which propagates straight out of the async generator into run_chat_task's generic except Exception. So the whole chat task dies, not just the tool call.

The consequences that make this worse than a one-off 500:

Every turn after it fails identically until the model happens to produce a fully-contained tool call. In the affected chat, eleven consecutive assistant turns ended in the exact same Unterminated string error block. The user sees the model "reply" with a wall of the same error and cannot get work done; each failed turn consumes a full model call including all the reasoning tokens.
Each crash persists an error turn into the chat tree, so the history accumulates a dozen > **Error:** Unterminated string... branches that then ride along in the context for every subsequent attempt, burning tokens and burying the actual work.
It is invisible from the outside. There is no signal distinguishing "the provider truncated a tool call" from "the model crashed". The user cannot tell whether retrying will help.

Two things would fix this cleanly:

Guard the parse: strict json.loads, fall back to whatever lenient extraction exists, and on total failure fail that tool call with an error the model can see and retry, instead of raising out of the stream loop and killing the task.
Treat a stream that ends before the accumulated arguments parse as an incomplete stream (retry it), rather than parsing whatever partial bytes landed.

The today-workaround for anyone hitting this: nothing on the cptr side, it needs the guard. If your provider is a proxy, the cuts that produce truncated arguments show up as broken pipes or read timeouts in the proxy log, and a proxy that retries internally or keeps the connection alive makes it rare. That is mitigation, not a fix.

Originally created by @Classic298 on GitHub (Sep 21, 2026). Original GitHub issue: https://github.com/open-webui/computer/issues/300 Same root cause as #252 (the unguarded `json.loads`), but this is the sharper end of it: what happens when the accumulator is not just empty but **truncated mid-string by a cut stream**, and what it does to the chat. Filing separately because the blast radius is different from the "empty/short JSON" case there. What happened, on cptr 0.9.21 behind an OpenAI-compatible proxy: The proxy cut the SSE stream mid-argument (broken pipe on its side, logged at the same second as the crash). cptr had accumulated `arguments_json` by concatenating deltas, and the accumulation ended mid-string: `{"content": "\"\"\"Regression tests for ...` and then nothing. When a later chunk carried `finish_reason: tool_calls`, cptr ran `json.loads` on that truncated string and raised: ``` json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 13 (char 12) ``` at `json.loads(tc["arguments_json"])` inside `stream_openai_completions`, which propagates straight out of the async generator into `run_chat_task`'s generic `except Exception`. So the whole chat task dies, not just the tool call. The consequences that make this worse than a one-off 500: 1. **Every turn after it fails identically until the model happens to produce a fully-contained tool call.** In the affected chat, eleven consecutive assistant turns ended in the exact same `Unterminated string` error block. The user sees the model "reply" with a wall of the same error and cannot get work done; each failed turn consumes a full model call including all the reasoning tokens. 2. **Each crash persists an error turn into the chat tree**, so the history accumulates a dozen `> **Error:** Unterminated string...` branches that then ride along in the context for every subsequent attempt, burning tokens and burying the actual work. 3. **It is invisible from the outside.** There is no signal distinguishing "the provider truncated a tool call" from "the model crashed". The user cannot tell whether retrying will help. Two things would fix this cleanly: - Guard the parse: strict `json.loads`, fall back to whatever lenient extraction exists, and on total failure fail **that tool call** with an error the model can see and retry, instead of raising out of the stream loop and killing the task. - Treat a stream that ends before the accumulated arguments parse as an incomplete stream (retry it), rather than parsing whatever partial bytes landed. The today-workaround for anyone hitting this: nothing on the cptr side, it needs the guard. If your provider is a proxy, the cuts that produce truncated arguments show up as broken pipes or read timeouts in the proxy log, and a proxy that retries internally or keeps the connection alive makes it rare. That is mitigation, not a fix.

bsakaguchi commented

2026-10-18 14:03:51 +00:00

Author

Owner

@owui-terminator[bot] commented on GitHub (Sep 21, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

🟢 #252 bug: unguarded json.loads on streamed tool-call arguments crashes the request on empty/truncated JSON
This is the direct parent issue for the same bug class: unguarded json.loads on streamed tool-call arguments. Your report is the more specific cut-stream case where the accumulator is truncated mid-string, but it has the same crash point and failure mode.
by 1-kabir

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

@owui-terminator[bot] commented on GitHub (Sep 21, 2026):  🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟢 [#252](https://github.com/open-webui/computer/issues/252) **bug: unguarded json.loads on streamed tool-call arguments crashes the request on empty/truncated JSON** *This is the direct parent issue for the same bug class: unguarded `json.loads` on streamed tool-call arguments. Your report is the more specific cut-stream case where the accumulator is truncated mid-string, but it has the same crash point and failure mode.* *by 1-kabir* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.

bsakaguchi referenced this issue

2026-10-18 14:03:54 +00:00

[GH-ISSUE #306] bug: ACP agents (cursor, cline, gemini, grok) never finish a turn; the reply only appears on stop #219