mirror of
https://github.com/open-webui/computer.git
synced 2026-11-05 01:17:24 +00:00
[GH-ISSUE #277] bug: Compaction does not work and the chat continues where it left of #190
Labels
No labels
enhancement
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
open-webui/computer#190
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @JKGeovision on GitHub (Sep 15, 2026).
Original GitHub issue: https://github.com/open-webui/computer/issues/277
Hello,
I discovered that the compaction feature does not work correctly. Eventhough the compaction gets the model to return something the next message continues right where the last one left of. My assumption is that the compaction correctly gets the model to summaries but subsequent request dont cut off the chat history before that.
Thank you for your time and consideration.
Best regards,
Joshua Krimmer
@owui-terminator[bot] commented on GitHub (Sep 15, 2026):
🔍 Related Issues Found
I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:
🟢 #162 bug: repeated context compaction can drop every user message and break Qwen/vLLM
This is directly about automatic context compaction in cptr/computer and describes a failure mode after compaction where the next request is malformed due to how history is retained. The new issue also reports compaction succeeding superficially but the following chat continues with the old context instead of restarting properly.
by calderanicola
🟢 #209 bug: task silently marked complete when the post-compaction completion returns zero tokens
This issue is about the request immediately after compaction behaving incorrectly, with compaction followed by a bad continuation/termination outcome. It is closely related to post-compaction request handling, which is the core symptom in the new report.
by ThalynLabs
🟣 #146 bug: The /compact skill is not working
Although narrower and closed, it specifically reports that the
/compactfeature does not work at all, which overlaps with the user's claim that compaction appears ineffective. It may contain the same underlying compaction path or historical fix context.by JKGeovision
💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.
This comment was generated automatically. React with 👍 if helpful, 👎 if not.
@JKGeovision commented on GitHub (Sep 15, 2026):
Compaction ineffective against single-slot llama.cpp backend: largest conversation grows to context limit while other conversations do compact; internal calls time out and retry-storm the slot
Summary
When cptr runs against a local llama.cpp server (single slot, ~256k context), context compaction does not protect the largest conversation. In a captured llama.cpp log window (≈5 min) one conversation's context grew monotonically from 110,753 → 256,571 tokens (98 % of the model context) without a single compaction, although the default compaction threshold is 80,000 tokens.
At the same time, other, smaller conversations do trigger compaction (see evidence: two summarization requests with the exact
max_tokens=1000signature ofsummarize_messages()), and several requests from cptr are canceled client-side (http client error: Connection handling canceled, 5× within 31 s) — i.e. from the user's perspective cptr sends multiple concurrent requests, one of them grows until the context limit is hit, and another one times out every time.The log also shows the systemic consequence: 4–5 concurrent conversations competing for one slot cause prompt-cache thrashing (a single eviction of 15,366 MiB of KV cache) and full re-evaluation of 100k+ token prompts (215 s, 117 s, 114 s, 116 s), which in turn makes every queued internal call (summarize/review/approval) exceed its hard 15 s/20 s client timeouts.
This is cptr used natively (not via the OpenAI-compatible gateway) with llama.cpp configured as an OpenAI-compatible connection.
Environment
llama-server, single slot (id 0), model "Qwen3.8 27B", context 262,144 tokens, speculative decoding enabledllama-serveron port 55467 (log linessrv proxy_request: proxying request to model ... on port 55467)CHAT_COMPACT_TOKEN_THRESHOLD(80,000) assumed — please confirm the effective threshold in this setup (see "Diagnostics requested")Reproduction
delegate_tasksub-agents) so that several cptr conversations issue requests against the same slot.Observed: the large conversation is never compacted and keeps growing until the model context limit; internal LLM calls (summarization, reviews) time out; llama.cpp logs show client-side connection cancellations and massive KV-cache evictions.
Evidence from the llama.cpp log
1. The growing conversation is never compacted (110k → 256k, threshold 80k)
All requests of this conversation are selected by LCP prefix similarity (0.959–1.000), i.e. one conversation:
n_tokens)Already at the start of the window the conversation is 30 %+ above the 80k threshold, and it never drops — every iteration only appends (a few hundred to a few thousand new tokens). No summarization request attributable to this conversation appears anywhere in the window.
Raw excerpts:
2. Other conversations DO compact — but only by the exact
summarize_messages()signatureTwo independent requests in the window have the unmistakable signature of cptr's compaction summarization call: prompt of ~7.3–7.8k tokens (system prompt + the ≤30,000-char message dump built in
summarize_messages) and exactly 1,000 generated tokens (themax_tokens=1000cap passed togenerate_text— seeutils/summarize.py:75):So the compaction pipeline works in principle — but the largest conversation (the one most in need of compaction) is precisely the one it never fires for. The most plausible mechanism is the estimation fallback in
should_compact()(see Root cause 1): token-dense content underestimates real token counts by ~2×, so a real-110k-token dense conversation can still look like ~55–75k to the decision, while lighter (more natural-language) conversations of 80–90k tokens do cross the threshold.3. Client-side cancellations ("the request that always times out")
Five client disconnects are logged by the (front) server within ~31 s, each while requests of other conversations were still running/finishing:
4. Concurrent conversations + KV-cache thrashing
Besides the growing conversation, at least three further independent conversations are active in the same window (different prefix chains):
Because only one slot exists, these serialize, and the prompt cache cannot hold them all:
Result: the next request of an evicted conversation pays a full re-evaluation at ~200 tokens/s (215.6 s for 109,772 tokens; 117.5 s for 22,768; 114.0 s for 71,262; 116.7 s for 71,865). While one such monster request occupies the slot, every other in-flight cptr request sits in the queue — which is exactly when cptr's short internal-call deadlines (15 s/20 s) and stream read timeouts fire.
Root cause analysis (cptr 0.9.21 source)
RC1 (primary hypothesis) — Compaction decision degrades to a
len/4estimate when provider usage is absent/zero, which underestimates token-dense contentutils/context.py:33-60(should_compact):with
estimate_tokens(text) = max(1, len(text) // 4)(utils/context.py:8-10).Two problems:
utils/chat_task.py:2255-2283and2347-2356carry a dedicatedprovider_type == "llama.cpp" and usage_context_tokens(last_usage) <= 0branch that estimates context usage for the UI meter — but the compaction decision in the same loop does not get any equivalent protection; it silently runs on thelen/4estimate.Note: llama.cpp only emits
usagein the final stream chunk whenstream_options: {"include_usage": true}is set (cptr does send it —utils/ai.py:652), but any intermediary in the chain (llama-server proxy mode, older server builds, reverse proxies) can drop or rewrite that final chunk. Whether usage actually reaches cptr in this setup is the single most important thing to confirm (see Diagnostics).RC2 — Keep-zone is sized by message count, not tokens → no convergence guarantee
utils/chat_task.py:2135-2138:A recent single message can dominate the whole context: tool results are capped at 50,000 characters (
CHAT_TOOL_MAX_CHARS,env.py:63≈ 12.5–25k tokens depending on density),run_commandoutput at 8,000, user-pasted content is unbounded, and images are worth ~1k tokens each. If the newest 40 % contains such a message, compaction drops mostly small old messages, the context stays above the threshold, andshould_compact()fires again on the very next iteration — an LLM summarization call per iteration that never converges. There is no "after compaction we are still above threshold → also cap/truncate keep-zone messages" logic.RC3 — The summarization call has hard 15 s / 20 s deadlines that a busy local backend cannot meet → silent degradation to a 2,000-char stub
utils/summarize.py:65-83calls:generate_textwraps the call inasyncio.wait_for(..., timeout=20)(utils/ai.py:244, 287-300),chat_completionuseshttpx.AsyncClient(timeout=httpx.Timeout(15))— a hardcoded 15 s connect/read/write timeout (utils/ai.py:136).On a single-slot local backend, the summarization request first waits in the queue behind whatever long request is running (in the log: 30–300 s), then needs its own prompt eval. 15 s/20 s is not even a fraction of that. Every such call therefore:
Connection handling canceled(Evidence 3), and_naive_summary(utils/summarize.py:86-96): a 2,000-character stub of ≤200-char snippets per user/assistant message, which is then persisted as the conversation checkpoint (ChatMessage.update(checkpoint_message_id, chat_summary=summary),utils/chat_task.py:2150). Each failed pass overwrites the previous summary with a worse one — compaction silently degrades the conversation instead of protecting it.RC4 — Even on success, the summary is hard-capped at 1,000 tokens
utils/summarize.py:75passesmax_tokens=1000. Both summarization requests in the log hit the cap exactly (eval time ... / 1000 tokens), i.e. the summary was truncated mid-sentence. Combined with the input caps (≤2,000 chars per message, ≤30,000 chars total,utils/summarize.py:56-63), a drop zone of 100k+ tokens compresses to a 1,000-token stub.RC5 — No concurrency control for the many internal LLM calls against the same single slot
For a single user turn, cptr can issue, in addition to the main streaming request:
utils/chat_task.py:739-796(generate_json, 20 s deadline)asyncio.create_taskat end of turn (utils/chat_task.py:2842-2857→utils/memory.py:1574-1583), so it overlaps the next user turnutils/chat_task.py:2858-2877→utils/skills.py:776-781)utils/chat_task.py:799-846)…all against the same single-slot backend, plus every other open chat or
delegate_tasksub-agent. There is no queue, semaphore, or priority between the interactive main request and these internal calls. In the log this manifests as the multiple concurrent conversations and the burst of client-side cancellations.RC6 — Stream retries re-send the entire request up to 3× (thundering herd on one slot)
utils/ai.py:56-58, 816-829: on any retryable stream error before the first event — includinghttpx.ReadTimeout, which is what a client-side timeout looks like when the first attempt is still queued/processing on the backend — the full request body (i.e. the whole, potentially 100k+ token context) is re-sent, up to 3 times with only 0.5–1.5 s backoff. On a single slot this adds full-context re-evaluations to an already saturated queue instead of recovering.Expected behavior
compact_token_thresholdshould be compacted before it approaches the model context limit, and compaction should converge (post-compaction context below threshold).Proposed fixes
usagevsestimate) at INFO level so "why didn't it compact" is answerable from cptr logs.provider_type == "llama.cpp", treat "no usage in stream" as a known degraded mode and lower the effective threshold (or surface a warning) rather than silently estimating.chat.context_compaction.timeout_secondsin config, default ≥ 600 s for non-cloud providers).max_tokensfor summarization (e.g. 2,000–4,000) or make it proportional to the drop-zone size, so the checkpoint summary is not truncated mid-sentence.Diagnostics requested (to confirm RC1 vs. alternatives)
chat.compact_token_thresholdin admin UI app_config /[chat] compact_token_thresholdinconfig.toml/CHAT_COMPACT_TOKEN_THRESHOLDenv (expected: 80000 default).CPTR_LOG_LEVEL=DEBUG, especially:compacted:(are there any for the growing conversation?),[summarize] LLM call failed, using naive fallback(RC3 occurrences),[task …] db save/ iteration counts."stream_options": {"include_usage": true}and check whether the final chunk containsusage: If the final chunk has nousage(orprompt_tokens: 0), the compaction decision runs on thelen/4estimate — confirming RC1.--no-warmup, etc.).delegate_tasksub-agents) to map log tasks 62723/71618 to their parent conversations.Workarounds (until fixed)
compact_token_thresholdwell below the model context (e.g. 40–60k) and keep conversations with dense tool output shorter.