open-webui/computer

Fork 0

mirror of https://github.com/open-webui/computer.git synced 2026-11-05 01:17:24 +00:00

[GH-ISSUE #193] bug: `stream` defaults to `true`, so non-streaming OpenAI-compatible clients receive SSE and fail #113

New issue

Closed

opened 2026-10-18 14:03:03 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-18 14:03:03 +00:00

Owner

Originally created by @penradioGables on GitHub (Jul 28, 2026).
Original GitHub issue: https://github.com/open-webui/computer/issues/193

Summary

ChatCompletionRequest.stream defaults to True in routers/gateway.py. The OpenAI API specifies stream as defaulting to false when the field is omitted, so any client that wants a single JSON response — and therefore omits the field — silently receives an event-stream instead, and fails to parse it.

The agent itself answers correctly; only the transport framing is wrong.

Version: cptr 0.9.10, Python 3.12, SQLite backend. Reproduced against a running instance.

Root cause

routers/gateway.py:

class ChatCompletionRequest(BaseModel):
    model: str
    messages: list[dict]
    stream: bool = True        # <-- OpenAI's documented default is false

The dispatch below it is correct, and the non-streaming branch is fully implemented:

    if body.stream:
        return StreamingResponse(
            _stream(output_queue, completion_id, created, body.model),
            media_type="text/event-stream",
            ...
        )
    else:
        # Non-streaming: collect all text, return as single response
        full_text = await _collect(output_queue)
        return {
            "id": completion_id,
            "object": "chat.completion",
            ...
        }

_collect() drains the queue, handles delta / output / done / error events, and returns a well-formed chat.completion object. The else: branch appears correct — it is simply unreachable in practice, because real clients omit stream rather than sending false explicitly.

Reproduction

# Fails: returns an SSE body where a JSON object is expected
curl -X POST http://<cptr>:8000/v1/chat/completions \
  -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
  -d '{"model":"cptr/cptr","messages":[{"role":"user","content":"hi"}]}'

# Works: HTTP 200
curl -X POST http://<cptr>:8000/v1/chat/completions \
  -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
  -d '{"model":"cptr/cptr","stream":true,"messages":[{"role":"user","content":"hi"}]}'

Behind a proxy the failure surfaces as a 500. Via LiteLLM:

litellm.InternalServerError: OpenAIException - Empty or invalid response from LLM
endpoint. Received: 'data: {"id": "chatcmpl-...", "object": "chat.completion.chunk",
"model": "cptr/cptr", "choices": [{"index": 0, "delta": {"role": "assistant",
"content": ""}, "finish_reason": null}]}
...

The correct answer is visible inside that unparsed stream.

There is no client-side workaround

Setting "stream": false explicitly does not help when a proxy sits in between. Verified against LiteLLM 1.95.0: an explicit "stream": false in the client request still produces the same failure, because LiteLLM omits the field on the wire for non-streaming calls rather than forwarding false. The caller cannot reach the else: branch at all.

Suggested fix

 class ChatCompletionRequest(BaseModel):
     model: str
     messages: list[dict]
-    stream: bool = True
+    stream: bool = False

This should be safe for existing consumers: Open WebUI sets stream: true explicitly, so its behaviour is unchanged. Only clients that currently omit the field are affected, and those are the ones that are broken today.

Secondary, in the same branch

The non-streaming response hardcodes token counts to zero:

            "usage": {
                "prompt_tokens": 0,
                "completion_tokens": 0,
                "total_tokens": 0,
            },

This is currently invisible because the branch is unreachable. Once the default is corrected it becomes live, and any proxy or client doing token accounting off usage will record zeros for every non-streaming call. The streaming path already surfaces real counts.

Happy to open a PR for the one-line change if that is useful.

Originally created by @penradioGables on GitHub (Jul 28, 2026). Original GitHub issue: https://github.com/open-webui/computer/issues/193 ### Summary `ChatCompletionRequest.stream` defaults to `True` in `routers/gateway.py`. The OpenAI API specifies `stream` as defaulting to **`false`** when the field is omitted, so any client that wants a single JSON response — and therefore omits the field — silently receives an event-stream instead, and fails to parse it. The agent itself answers correctly; only the transport framing is wrong. **Version:** cptr 0.9.10, Python 3.12, SQLite backend. Reproduced against a running instance. ### Root cause `routers/gateway.py`: ```python class ChatCompletionRequest(BaseModel): model: str messages: list[dict] stream: bool = True # <-- OpenAI's documented default is false ``` The dispatch below it is correct, and the non-streaming branch is fully implemented: ```python if body.stream: return StreamingResponse( _stream(output_queue, completion_id, created, body.model), media_type="text/event-stream", ... ) else: # Non-streaming: collect all text, return as single response full_text = await _collect(output_queue) return { "id": completion_id, "object": "chat.completion", ... } ``` `_collect()` drains the queue, handles `delta` / `output` / `done` / `error` events, and returns a well-formed `chat.completion` object. The `else:` branch appears correct — it is simply unreachable in practice, because real clients omit `stream` rather than sending `false` explicitly. ### Reproduction ```bash # Fails: returns an SSE body where a JSON object is expected curl -X POST http://<cptr>:8000/v1/chat/completions \ -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \ -d '{"model":"cptr/cptr","messages":[{"role":"user","content":"hi"}]}' # Works: HTTP 200 curl -X POST http://<cptr>:8000/v1/chat/completions \ -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \ -d '{"model":"cptr/cptr","stream":true,"messages":[{"role":"user","content":"hi"}]}' ``` Behind a proxy the failure surfaces as a 500. Via LiteLLM: ``` litellm.InternalServerError: OpenAIException - Empty or invalid response from LLM endpoint. Received: 'data: {"id": "chatcmpl-...", "object": "chat.completion.chunk", "model": "cptr/cptr", "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": null}]} ... ``` The correct answer is visible inside that unparsed stream. ### There is no client-side workaround Setting `"stream": false` explicitly does not help when a proxy sits in between. Verified against LiteLLM 1.95.0: an explicit `"stream": false` in the client request still produces the same failure, because LiteLLM omits the field on the wire for non-streaming calls rather than forwarding `false`. The caller cannot reach the `else:` branch at all. ### Suggested fix ```diff class ChatCompletionRequest(BaseModel): model: str messages: list[dict] - stream: bool = True + stream: bool = False ``` This should be safe for existing consumers: Open WebUI sets `stream: true` explicitly, so its behaviour is unchanged. Only clients that currently omit the field are affected, and those are the ones that are broken today. ### Secondary, in the same branch The non-streaming response hardcodes token counts to zero: ```python "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0, }, ``` This is currently invisible because the branch is unreachable. Once the default is corrected it becomes live, and any proxy or client doing token accounting off `usage` will record zeros for every non-streaming call. The streaming path already surfaces real counts. Happy to open a PR for the one-line change if that is useful.

bsakaguchi closed this issue

2026-10-18 14:03:03 +00:00

bsakaguchi commented

2026-10-18 14:03:04 +00:00

Author

Owner

@owui-terminator[bot] commented on GitHub (Jul 28, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

🟣 #96 bug: OpenAI-compatible streaming usage can end responses early
Related only at the broad component level: it also concerns OpenAI-compatible chat/completions behavior in the same codepath, but the bug is about premature termination on streaming usage events, not the default stream parameter or SSE vs JSON framing.
by sandrogauci

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

@owui-terminator[bot] commented on GitHub (Jul 28, 2026):  🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟣 [#96](https://github.com/open-webui/computer/issues/96) **bug: OpenAI-compatible streaming usage can end responses early** *Related only at the broad component level: it also concerns OpenAI-compatible chat/completions behavior in the same codepath, but the bug is about premature termination on streaming `usage` events, not the default `stream` parameter or SSE vs JSON framing.* *by sandrogauci* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.

bsakaguchi commented

2026-10-18 14:03:04 +00:00

Author

Owner

@tjbck commented on GitHub (Jul 31, 2026):

Should be addressed!

@tjbck commented on GitHub (Jul 31, 2026): Should be addressed!

No labels

enhancement

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

open-webui/computer#113

No description provided.

Rows
Columns

[GH-ISSUE #193] bug: stream defaults to true, so non-streaming OpenAI-compatible clients receive SSE and fail #113

Summary

Root cause

Reproduction

There is no client-side workaround

Suggested fix

Secondary, in the same branch

[GH-ISSUE #193] bug: `stream` defaults to `true`, so non-streaming OpenAI-compatible clients receive SSE and fail #113