open-webui/computer

Fork 0

mirror of https://github.com/open-webui/computer.git synced 2026-11-05 01:17:24 +00:00

[GH-ISSUE #282] Memory filter false-positives silently drop ALL recalled memory, trivially bypassed, and rejects legitimate saves #193

New issue

Open

opened 2026-10-18 14:03:43 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-18 14:03:43 +00:00

Owner

Originally created by @Classic298 on GitHub (Sep 20, 2026).
Original GitHub issue: https://github.com/open-webui/computer/issues/282

What happens

Memory with the phrase "system prompt" in it — anywhere, in any legitimate sentence — gets treated as a prompt-injection attack. Two consequences:

Legitimate memories can't be saved. Saving My team reviews system prompt designs is rejected at write time with memory content looks like prompt-injection or credential-exfiltration instructions.
Existing memory silently vanishes from the agent. The check runs again when memory is loaded into the prompt. It is all-or-nothing per file: one match anywhere and the entire file is replaced with a single [BLOCKED: ...] line. A 20KB memory index built over months gets silently replaced by one sentence, on every turn, with no warning to the user and no log entry. The memory looks intact in the UI (the Settings > Memory view reads the file directly), so there's no way to notice except the agent losing all its saved knowledge.

Verified on v0.9.21 with both scenarios (rejection at write time, and a baseline file with one matching line producing the [BLOCKED: ...] placeholder at prompt build).

The filter is trivially bypassed

The check is a fixed phrase list (in cptr/utils/memory.py, PROMPT_THREAT_RE). Every phrase needs the literal space: system prompt, ignore previous instructions, exfiltrate, etc. An actual attacker paraphrases ("disregard what you were told above") and passes straight through, because the filter only catches verbatim phrasings.

So the trade is: blocks the clumsy, passes the deliberate, and silently destroys legitimate memory. A phrase blocklist with file-level blast radius punishes exactly the users who keep real notes — security researchers, devs documenting injection tests, anyone discussing prompts at all.

Repro

cptr → Settings → Memory → add an entry containing the phrase system prompt → rejected.
Add it via the file (works, same failure once loaded): put that sentence into ~/.cptr/memory/users/<id>/USER.md directly, ask the agent what it remembers. All of it is gone, replaced by the BLOCKED placeholder.

Suggested fix

Filter per section/entry instead of per file, so a false positive costs one entry instead of the whole memory, and surface a warning in the UI when content gets blocked. For actual injection defence, provenance framing (label injected memory as untrusted data the model should treat as information, not instructions) works against paraphrased attacks too; a phrase blocklist does not.

As a today-solution, anyone who hits this can hyphenate the phrase (system-prompt) or move it into a vault sub-file (only the baseline file gets the whole-file treatment).

Originally created by @Classic298 on GitHub (Sep 20, 2026). Original GitHub issue: https://github.com/open-webui/computer/issues/282 ## What happens Memory with the phrase "system prompt" in it — anywhere, in any legitimate sentence — gets treated as a prompt-injection attack. Two consequences: 1. **Legitimate memories can't be saved.** Saving `My team reviews system prompt designs` is rejected at write time with `memory content looks like prompt-injection or credential-exfiltration instructions`. 2. **Existing memory silently vanishes from the agent.** The check runs again when memory is loaded into the prompt. It is all-or-nothing per file: one match anywhere and the entire file is replaced with a single `[BLOCKED: ...]` line. A 20KB memory index built over months gets silently replaced by one sentence, on every turn, with no warning to the user and no log entry. The memory looks intact in the UI (the Settings > Memory view reads the file directly), so there's no way to notice except the agent losing all its saved knowledge. Verified on v0.9.21 with both scenarios (rejection at write time, and a baseline file with one matching line producing the `[BLOCKED: ...]` placeholder at prompt build). ## The filter is trivially bypassed The check is a fixed phrase list (in `cptr/utils/memory.py`, `PROMPT_THREAT_RE`). Every phrase needs the literal space: `system prompt`, `ignore previous instructions`, `exfiltrate`, etc. An actual attacker paraphrases ("disregard what you were told above") and passes straight through, because the filter only catches verbatim phrasings. So the trade is: blocks the clumsy, passes the deliberate, and silently destroys legitimate memory. A phrase blocklist with file-level blast radius punishes exactly the users who keep real notes — security researchers, devs documenting injection tests, anyone discussing prompts at all. ## Repro 1. `cptr` → Settings → Memory → add an entry containing the phrase `system prompt` → rejected. 2. Add it via the file (works, same failure once loaded): put that sentence into `~/.cptr/memory/users/<id>/USER.md` directly, ask the agent what it remembers. All of it is gone, replaced by the BLOCKED placeholder. ## Suggested fix Filter per section/entry instead of per file, so a false positive costs one entry instead of the whole memory, and surface a warning in the UI when content gets blocked. For actual injection defence, provenance framing (label injected memory as untrusted data the model should treat as information, not instructions) works against paraphrased attacks too; a phrase blocklist does not. As a today-solution, anyone who hits this can hyphenate the phrase (`system-prompt`) or move it into a vault sub-file (only the baseline file gets the whole-file treatment).

bsakaguchi commented

2026-10-18 14:03:43 +00:00

Author

Owner

@owui-terminator[bot] commented on GitHub (Sep 20, 2026):

⚠️ Missing Issue Title Prefix

@Classic298, your issue title is missing a categorising prefix (e.g. bug:, feat:, docs:).

Please update the title to lead with one of:

bug: bug report or error
feat: feature request or enhancement
docs: documentation issue
question: usage question
help: support request

Example: bug: Login fails when the password contains special characters

@owui-terminator[bot] commented on GitHub (Sep 20, 2026): # ⚠️ Missing Issue Title Prefix @Classic298, your issue title is missing a categorising prefix (e.g. `bug:`, `feat:`, `docs:`). Please update the title to lead with one of: - **bug**: bug report or error - **feat**: feature request or enhancement - **docs**: documentation issue - **question**: usage question - **help**: support request Example: `bug: Login fails when the password contains special characters`

bsakaguchi commented

2026-10-18 14:03:43 +00:00

Author

Owner

@owui-terminator[bot] commented on GitHub (Sep 20, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

🟣 #12 feat: AGENT.md / CLAUDE.md / MEMORY.md
This issue introduced the MEMORY.md mechanism and states it is loaded as additional system prompt on every request, which is the same memory-loading path affected by the blocking behavior described here.
by Classic298

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

@owui-terminator[bot] commented on GitHub (Sep 20, 2026):  🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟣 [#12](https://github.com/open-webui/computer/issues/12) **feat: AGENT.md / CLAUDE.md / MEMORY.md** *This issue introduced the MEMORY.md mechanism and states it is loaded as additional system prompt on every request, which is the same memory-loading path affected by the blocking behavior described here.* *by Classic298* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.

bsakaguchi commented

2026-10-18 14:03:43 +00:00

Author

Owner

@Classic298 commented on GitHub (Oct 12, 2026):

@tjbck could you take this one first? Still happens on 0.9.21.

Today it blocked my entire memory index (about 18 KB) because of a single sentence containing "system prompt". The session started with none of my saved rules, and nothing in the UI showed that anything was dropped.

Smallest fix: filter per section instead of per file, so one match costs one section and the rest still loads.

@Classic298 commented on GitHub (Oct 12, 2026): @tjbck could you take this one first? Still happens on 0.9.21. Today it blocked my entire memory index (about 18 KB) because of a single sentence containing "system prompt". The session started with none of my saved rules, and nothing in the UI showed that anything was dropped. Smallest fix: filter per section instead of per file, so one match costs one section and the rest still loads.

No labels

enhancement

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

open-webui/computer#193

No description provided.

Rows
Columns