open-webui/computer

Fork 0

mirror of https://github.com/open-webui/computer.git synced 2026-11-05 01:17:24 +00:00

[GH-ISSUE #317] Anthropic requests are silently capped at a hardcoded max_tokens of 4096: long replies truncate mid-sentence and are saved as complete, no setting exists #229

New issue

Open

opened 2026-10-18 14:03:59 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-18 14:03:59 +00:00

Owner

Originally created by @Classic298 on GitHub (Oct 12, 2026).
Original GitHub issue: https://github.com/open-webui/computer/issues/317

What happens

Long replies from Claude models are silently cut off mid-sentence. The reply just ends, the turn is saved as complete, and there is no error, no truncation notice, nothing. Agentic turns with tool calls are worse: the cut can land inside a tool call's arguments, which then fails to parse and can kill the whole task. It looks like the model randomly decided to stop.

Why

The Anthropic streaming path hardcodes an output cap and sends it on every request: max_tokens: 4096. There is no setting for it anywhere in the UI or config schema, so every user of the Anthropic provider gets this cap whether they want it or not. When the model hits the cap, the stream ends with the stop reason max_tokens, which cptr also does not read (see #316), so the truncated reply is stored as a normal finished message.

This is inconsistent with the other providers: the OpenAI chat completions path sends no max token cap by default, and the Responses path sends none either. Only Anthropic gets a forced cap.

Expected versus actual

Expected: no default output cap. All major APIs treat the output limit as optional; omitting max_tokens means the model runs to its own limit, and providers that do require it (older Anthropic models) can be configured per model.
Actual: every Anthropic streaming request is silently capped at 4096 output tokens with no way to change it except hand-editing model config to inject a request param.

Repro

Any chat with an Anthropic connection and a Claude model.
Ask for something long enough to exceed 4096 output tokens, for example a detailed multi-file walkthrough or any verbose agentic turn with thinking enabled.
The reply ends mid-sentence at exactly 4096 output tokens (visible in the stored usage of the saved message), saved as done with no indication of truncation.

Consequences

Long replies are always truncated at 4096 tokens, invisible to the user.
Thinking models lose a large chunk of the budget to reasoning tokens, so effective reply length is even shorter than 4096.
Tool-call arguments can be cut off mid-JSON, which downstream parsing then treats as a malformed call.
Any user who wants a different cap has no setting; the only workaround is knowing that per-model request params override the hardcoded body.

Fix direction

Remove the hardcoded cap from the request body and let the provider default apply. If a cap is desired for cost protection, make it a per-model setting in the same place the other per-model request params live, not a silent constant. And per #316, the stop reason should be captured so a max_tokens stop is at least visible in the chat instead of silent.

Related: #316 (stop reasons in general are never captured, refusals and max-token stops both vanish).

Workaround for now

Per-model request params are merged after the hardcoded body, so setting a max_tokens request param for the model in config does change what is sent. That fixes the size for users who know about it, but it is undiscoverable and the default remains wrong.

Originally created by @Classic298 on GitHub (Oct 12, 2026). Original GitHub issue: https://github.com/open-webui/computer/issues/317 ## What happens Long replies from Claude models are silently cut off mid-sentence. The reply just ends, the turn is saved as complete, and there is no error, no truncation notice, nothing. Agentic turns with tool calls are worse: the cut can land inside a tool call's arguments, which then fails to parse and can kill the whole task. It looks like the model randomly decided to stop. ## Why The Anthropic streaming path hardcodes an output cap and sends it on every request: `max_tokens: 4096`. There is no setting for it anywhere in the UI or config schema, so every user of the Anthropic provider gets this cap whether they want it or not. When the model hits the cap, the stream ends with the stop reason `max_tokens`, which cptr also does not read (see #316), so the truncated reply is stored as a normal finished message. This is inconsistent with the other providers: the OpenAI chat completions path sends no max token cap by default, and the Responses path sends none either. Only Anthropic gets a forced cap. ## Expected versus actual Expected: no default output cap. All major APIs treat the output limit as optional; omitting `max_tokens` means the model runs to its own limit, and providers that do require it (older Anthropic models) can be configured per model. Actual: every Anthropic streaming request is silently capped at 4096 output tokens with no way to change it except hand-editing model config to inject a request param. ## Repro 1. Any chat with an Anthropic connection and a Claude model. 2. Ask for something long enough to exceed 4096 output tokens, for example a detailed multi-file walkthrough or any verbose agentic turn with thinking enabled. 3. The reply ends mid-sentence at exactly 4096 output tokens (visible in the stored usage of the saved message), saved as done with no indication of truncation. ## Consequences - Long replies are always truncated at 4096 tokens, invisible to the user. - Thinking models lose a large chunk of the budget to reasoning tokens, so effective reply length is even shorter than 4096. - Tool-call arguments can be cut off mid-JSON, which downstream parsing then treats as a malformed call. - Any user who wants a different cap has no setting; the only workaround is knowing that per-model request params override the hardcoded body. ## Fix direction Remove the hardcoded cap from the request body and let the provider default apply. If a cap is desired for cost protection, make it a per-model setting in the same place the other per-model request params live, not a silent constant. And per #316, the stop reason should be captured so a `max_tokens` stop is at least visible in the chat instead of silent. Related: #316 (stop reasons in general are never captured, refusals and max-token stops both vanish). ## Workaround for now Per-model request params are merged after the hardcoded body, so setting a `max_tokens` request param for the model in config does change what is sent. That fixes the size for users who know about it, but it is undiscoverable and the default remains wrong.

bsakaguchi commented

2026-10-18 14:04:00 +00:00

Author

Owner

@owui-terminator[bot] commented on GitHub (Oct 12, 2026):

⚠️ Missing Issue Title Prefix

@Classic298, your issue title is missing a categorising prefix (e.g. bug:, feat:, docs:).

Please update the title to lead with one of:

bug: bug report or error
feat: feature request or enhancement
docs: documentation issue
question: usage question
help: support request

Example: bug: Login fails when the password contains special characters

@owui-terminator[bot] commented on GitHub (Oct 12, 2026): # ⚠️ Missing Issue Title Prefix @Classic298, your issue title is missing a categorising prefix (e.g. `bug:`, `feat:`, `docs:`). Please update the title to lead with one of: - **bug**: bug report or error - **feat**: feature request or enhancement - **docs**: documentation issue - **question**: usage question - **help**: support request Example: `bug: Login fails when the password contains special characters`

bsakaguchi commented

2026-10-18 14:04:00 +00:00

Author

Owner

@owui-terminator[bot] commented on GitHub (Oct 12, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

🟢 #316 Anthropic refusals are saved as successful empty replies: stop reason is never captured, turn looks blank and done, nothing in the logs
This is directly related: it documents the same Anthropic streaming path dropping the final stop reason, which causes max_tokens stops to look like normal completions. That omission is a core part of why capped Anthropic replies are silently saved as finished messages.
by Classic298
🟢 #300 Critical: bug: a cut stream leaves truncated tool-call arguments, json.loads on the accumulator kills the whole chat task and persists an error turn every retry
This is related as a downstream failure mode of the same truncation problem. It shows that when a stream is cut mid-tool-call, the accumulated arguments become invalid JSON and can crash the chat task.
by Classic298
🟢 #252 bug: unguarded json.loads on streamed tool-call arguments crashes the request on empty/truncated JSON
This is closely related because it covers the generic bug of streaming empty or truncated tool-call arguments causing json.loads failures. The new issue's Anthropic token cap can create exactly that kind of truncated arguments payload.
by 1-kabir

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

@owui-terminator[bot] commented on GitHub (Oct 12, 2026):  🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟢 [#316](https://github.com/open-webui/computer/issues/316) **Anthropic refusals are saved as successful empty replies: stop reason is never captured, turn looks blank and done, nothing in the logs** *This is directly related: it documents the same Anthropic streaming path dropping the final stop reason, which causes `max_tokens` stops to look like normal completions. That omission is a core part of why capped Anthropic replies are silently saved as finished messages.* *by Classic298* 2. 🟢 [#300](https://github.com/open-webui/computer/issues/300) **Critical: bug: a cut stream leaves truncated tool-call arguments, json.loads on the accumulator kills the whole chat task and persists an error turn every retry** *This is related as a downstream failure mode of the same truncation problem. It shows that when a stream is cut mid-tool-call, the accumulated arguments become invalid JSON and can crash the chat task.* *by Classic298* 3. 🟢 [#252](https://github.com/open-webui/computer/issues/252) **bug: unguarded json.loads on streamed tool-call arguments crashes the request on empty/truncated JSON** *This is closely related because it covers the generic bug of streaming empty or truncated tool-call arguments causing `json.loads` failures. The new issue's Anthropic token cap can create exactly that kind of truncated arguments payload.* *by 1-kabir* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.

No labels

enhancement

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

open-webui/computer#229

No description provided.

Rows
Columns