open-webui/computer

Fork 0

mirror of https://github.com/open-webui/computer.git synced 2026-11-05 01:17:24 +00:00

[GH-ISSUE #164] bug: problematic cache reuse #87

New issue

Open

opened 2026-10-18 14:02:47 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-18 14:02:47 +00:00

Owner

Originally created by @cclecle on GitHub (Jul 23, 2026).
Original GitHub issue: https://github.com/open-webui/computer/issues/164

Bug

On moderate to long conversation, I noticed that the inference engine cache is not being used efficiently .
Turns becomes longer and longer, and backend metrics and logs confirms it.

Reproduction

Run a long multi-turn conversation and measure speed + check you backend logs, notice mostly full prompt reprocessing at every turn, including in between tool calls.

Likely Root cause

In prompt_templates, {{MEMORY}} and maybe others fields like {{FILE_TREE}} are recomputed and alterated at each turns.
It modifies the conversation history, and the backend is loosing most of its cache because of the early divergence.

Suggestions

These fields should probably be frozen for the current discussion, and if the Agent needs update:

they can use tool call to retrieve updated values
and / or the harness can inject updated value at the bottom of the conversation rather than alterating the prefix

Environment

cptr 0.9.13
llamacpp

_Disclosure: this issue was investigated and root-caused with the help of ChatGPT , and is written and submitted on my behalf at my request.

Originally created by @cclecle on GitHub (Jul 23, 2026). Original GitHub issue: https://github.com/open-webui/computer/issues/164 ## Bug On moderate to long conversation, I noticed that the inference engine cache is not being used efficiently . Turns becomes longer and longer, and backend metrics and logs confirms it. ## Reproduction Run a long multi-turn conversation and measure speed + check you backend logs, notice mostly full prompt reprocessing at every turn, including in between tool calls. ## Likely Root cause In prompt_templates, {{MEMORY}} and maybe others fields like {{FILE_TREE}} are recomputed and alterated at each turns. It modifies the conversation history, and the backend is loosing most of its cache because of the early divergence. ## Suggestions These fields should probably be frozen for the current discussion, and if the Agent needs update: - they can use tool call to retrieve updated values - and / or the harness can inject updated value at the bottom of the conversation rather than alterating the prefix ## Environment - cptr 0.9.13 - llamacpp --- _Disclosure: this issue was investigated and root-caused with the help of ChatGPT , and is written and submitted on my behalf at my request.

bsakaguchi commented

2026-10-18 14:02:48 +00:00

Author

Owner

@owui-terminator[bot] commented on GitHub (Jul 23, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

🟢 #162 bug: repeated context compaction can drop every user message and break Qwen/vLLM
Both issues concern long-running agent conversations getting worse over time because the message sequence sent to the backend changes in ways that hurt correctness/efficiency. Issue #162 is about repeated compaction altering the retained conversation history; #164 is about changing prompt template fields like MEMORY/FILE_TREE and losing cache reuse due to early prompt divergence.
by calderanicola
🟢 #13 feat: tool execution policies / allow once, always allow, "auto mode"
This feature request discusses an 'auto mode' task model that gets context from the conversation and is explicitly 'cache optimized for cheap task model cache hits.' That makes it relevant to #164's concern about preserving stable prefixes and improving inference cache reuse across turns.
by Classic298

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

@owui-terminator[bot] commented on GitHub (Jul 23, 2026):  🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟢 [#162](https://github.com/open-webui/computer/issues/162) **bug: repeated context compaction can drop every user message and break Qwen/vLLM** *Both issues concern long-running agent conversations getting worse over time because the message sequence sent to the backend changes in ways that hurt correctness/efficiency. Issue #162 is about repeated compaction altering the retained conversation history; #164 is about changing prompt template fields like MEMORY/FILE_TREE and losing cache reuse due to early prompt divergence.* *by calderanicola* 2. 🟢 [#13](https://github.com/open-webui/computer/issues/13) **feat: tool execution policies / allow once, always allow, "auto mode"** *This feature request discusses an 'auto mode' task model that gets context from the conversation and is explicitly 'cache optimized for cheap task model cache hits.' That makes it relevant to #164's concern about preserving stable prefixes and improving inference cache reuse across turns.* *by Classic298* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.

No labels

enhancement

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

open-webui/computer#87

No description provided.

Rows
Columns