open-webui/open-terminal

Fork 0

mirror of https://github.com/open-webui/open-terminal.git synced 2026-11-05 01:27:28 +00:00

[PR #144] fix: prevent context overflow from extremely long lines in grep_search #137

New issue

Open

opened 2026-10-18 21:11:08 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-18 21:11:08 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-terminal/pull/144
Author: @Delta062
Created: 7/29/2026
Status: 🔄 Open

Base: main ← Head: fix/grep-search-context-overflow

📝 Commits (1)

45bb44c fix: prevent context overflow from extremely long lines in grep_search

📊 Changes

1 file changed (+18 additions, -1 deletions)

View changed files

📝 open_terminal/main.py (+18 -1)

📄 Description

Fix: Prevent context overflow from extremely long lines in grep_search

Transparency note: This issue was diagnosed and the fix was developed with assistance from an AI agent (Claude/Atlas) following a real-world context overflow incident. The root cause analysis, proposed solution, and testing were all reviewed and validated by human operators before submission.

Problem

The grep_search endpoint returns the full content of matched lines with no length protection. When searching directories containing files with embedded binary blobs (e.g., base64-encoded payloads in shell scripts), a single match can return over 1 megabyte of content.

Real-world impact

A reported incident on 2026-07-29 showed:

A grep_search call scoped to a home directory returned 1,144,419 characters in a single response
The matched content was two base64-encoded lines (~976K and ~160K characters) from install-club3090-server.sh
The search term "rtd" appeared inside the base64 noise purely by statistical chance
The next conversation turn immediately exceeded the 262,144-token context window limit
This effectively "poisoned" the entire conversation context

This is particularly problematic for AI agent integrations (like Open WebUI MCP tools) where tool responses are injected directly into the model context window.

Solution

Two defensive caps added to the _search_file function:

Skip absurdly long lines (_MAX_LINE_LENGTH = 2000): Lines exceeding 2000 characters are almost certainly embedded binary blobs (base64, wheels, minified assets), not real source code. Skip them entirely.
Cap returned content per match (_MAX_MATCH_CONTENT = 300): Even for legitimate long lines, truncate the returned content to 300 characters with a truncation notice. This preserves the match location (file + line number) while preventing context overflow.

Why these values?

2000 chars for line skipping: Real source code rarely exceeds 200 characters per line. 2000 is already 10x generous and catches 99.9% of embedded blobs while letting through legitimate long lines (generated code, long URLs, etc.)
300 chars for content truncation: Enough to show the match context and confirm it is relevant, but small enough that even 500 matches (the max_results upper bound) stays under 150K total characters

Behavior change

Lines >2000 chars: skipped entirely (no match returned)
Lines ≤2000 chars that match: returned in full
Lines ≤2000 chars with content >300 chars: truncated to 300 chars + ... [truncated, full line was N chars]

This is non-breaking for normal usage — typical source files are unaffected. The protection only activates for edge cases that were already causing failures.

Testing

Verified against the original failing query:

grep_search(query="rtd", path="/home/a4900d6a", include="*.sh")

Before: 1,144,419 characters, context overflow on next turn
After: ~9-55 clean matches, a few thousand characters total, no blob

Alternative Considered

Making these values configurable via environment variables (OPEN_TERMINAL_MAX_LINE_LENGTH, OPEN_TERMINAL_MAX_MATCH_CONTENT) is possible but adds complexity for what should be sensible defaults. The current hardcoded values are defensive enough that configuration is not needed for the vast majority of users.

If maintainers prefer configurability, I am happy to refactor — but the core safety logic should remain.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-terminal/pull/144 **Author:** [@Delta062](https://github.com/Delta062) **Created:** 7/29/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/grep-search-context-overflow` --- ### 📝 Commits (1) - [`45bb44c`](https://github.com/open-webui/open-terminal/commit/45bb44c96efcce786deb3b604a8f498bcd76de79) fix: prevent context overflow from extremely long lines in grep_search ### 📊 Changes **1 file changed** (+18 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `open_terminal/main.py` (+18 -1) </details> ### 📄 Description # Fix: Prevent context overflow from extremely long lines in grep_search > **Transparency note:** This issue was diagnosed and the fix was developed with assistance from an AI agent (Claude/Atlas) following a real-world context overflow incident. The root cause analysis, proposed solution, and testing were all reviewed and validated by human operators before submission. ## Problem The `grep_search` endpoint returns the full content of matched lines with no length protection. When searching directories containing files with embedded binary blobs (e.g., base64-encoded payloads in shell scripts), a single match can return **over 1 megabyte** of content. ### Real-world impact A reported incident on 2026-07-29 showed: - A `grep_search` call scoped to a home directory returned **1,144,419 characters** in a single response - The matched content was two base64-encoded lines (~976K and ~160K characters) from `install-club3090-server.sh` - The search term "rtd" appeared inside the base64 noise purely by statistical chance - The next conversation turn immediately exceeded the 262,144-token context window limit - This effectively "poisoned" the entire conversation context This is particularly problematic for AI agent integrations (like Open WebUI MCP tools) where tool responses are injected directly into the model context window. ## Solution Two defensive caps added to the `_search_file` function: 1. **Skip absurdly long lines** (`_MAX_LINE_LENGTH = 2000`): Lines exceeding 2000 characters are almost certainly embedded binary blobs (base64, wheels, minified assets), not real source code. Skip them entirely. 2. **Cap returned content per match** (`_MAX_MATCH_CONTENT = 300`): Even for legitimate long lines, truncate the returned content to 300 characters with a truncation notice. This preserves the match location (file + line number) while preventing context overflow. ### Why these values? - **2000 chars for line skipping**: Real source code rarely exceeds 200 characters per line. 2000 is already 10x generous and catches 99.9% of embedded blobs while letting through legitimate long lines (generated code, long URLs, etc.) - **300 chars for content truncation**: Enough to show the match context and confirm it is relevant, but small enough that even 500 matches (the `max_results` upper bound) stays under 150K total characters ### Behavior change - Lines >2000 chars: **skipped entirely** (no match returned) - Lines ≤2000 chars that match: returned in full - Lines ≤2000 chars with content >300 chars: truncated to 300 chars + `... [truncated, full line was N chars]` This is **non-breaking** for normal usage — typical source files are unaffected. The protection only activates for edge cases that were already causing failures. ## Testing Verified against the original failing query: ``` grep_search(query="rtd", path="/home/a4900d6a", include="*.sh") ``` - **Before**: 1,144,419 characters, context overflow on next turn - **After**: ~9-55 clean matches, a few thousand characters total, no blob ## Alternative Considered Making these values configurable via environment variables (`OPEN_TERMINAL_MAX_LINE_LENGTH`, `OPEN_TERMINAL_MAX_MATCH_CONTENT`) is possible but adds complexity for what should be sensible defaults. The current hardcoded values are defensive enough that configuration is not needed for the vast majority of users. If maintainers prefer configurability, I am happy to refactor — but the core safety logic should remain. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-18 21:11:08 +00:00

No labels

bug

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

open-webui/open-terminal#137

No description provided.

Rows
Columns