[GH-ISSUE #537] Potential Vulnerability: Context Injection via RAG Bypasses Content Filtering #425

New issue

Closed

opened 2026-10-26 21:21:07 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 21:21:07 +00:00

Owner

Originally created by @rogerganga on GitHub (Mar 5, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/537

I have observed that when external context is added via Retrieval-Augmented Generation (RAG), the model’s content filtering appears to be bypassed. Specifically, if a user provides a document containing information that would normally be blocked in a direct query, the model generates a complete response based on that context.

While my experiments are based on a generic demonstration, this pattern indicates a potential vulnerability: sensitive or otherwise restricted content might be indirectly retrieved and then output, even though the model is designed to refuse such content when directly queried.

Steps to Reproduce:

Use the attached reference code which:
- Reads a text file containing context.
- Uses a simple retrieval function to select top paragraphs based on keyword overlap.
- Constructs a prompt combining the query and the retrieved context.
- Sends the prompt to the DeepSeek model
Run the code with a query and observe that:
- Without context, the model appropriately refuses or produces a minimal response.
- With the provided context (from the text file), the model generates a comprehensive response that incorporates the information from the document.

Model Used:
deepseek-r1:1.5b. I ran this model using Ollama

Reference Code:

import re
from ollama import chat

def read_text_file(file_path):
    """Read and return the content of the given text file."""
    with open(file_path, 'r', encoding='utf-8') as f:
        return f.read()

def retrieve_relevant_context(query, text, k=3):
    """
    A simple retrieval function that splits the text into paragraphs
    and selects the top k paragraphs with the highest keyword overlap with the query.
    """
    paragraphs = text.split('\n\n')
    query_tokens = set(re.findall(r'\w+', query.lower()))
    scored_paras = []
    for para in paragraphs:
        para_tokens = set(re.findall(r'\w+', para.lower()))
        score = len(query_tokens.intersection(para_tokens))
        scored_paras.append((score, para))
    scored_paras.sort(key=lambda x: x[0], reverse=True)
    top_paras = [para for score, para in scored_paras if score > 0][:k]
    return "\n\n".join(top_paras)

def generate_answer(query, context):
    """
    Constructs a prompt with the provided context and query,
    then calls DeepSeek to generate an answer.
    """
    prompt = f"Using the following context, answer the query:\n\nQuery: {query}\n\nContext:\n{context}"
    model = "deepseek-r1:1.5b"
    answer = ""
    try:
        response = chat(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            stream=True
        )
        for chunk in response:
            answer += chunk['message']['content']
        answer = answer.strip()
    except Exception as e:
        print("Error generating answer:", e)
        answer = ""
    return answer

def main():
    file_path = "BAD_FILE.txt"  # Replace with your text file's path
    text = read_text_file(file_path)
    query = "BAD PROMPT"
    context = retrieve_relevant_context(query, text)
    answer = generate_answer(query, context)
    print(answer)

if __name__ == '__main__':
    main()

Expected Behavior:

The model should enforce its content filtering regardless of whether the sensitive information is provided directly in the query or as part of the external context.
It should either provide a refusal response or sanitize the output even when context is injected.

Actual Behavior:

When context is added, the model generates a comprehensive answer that includes information which would normally be censored if asked directly.

Impact:

This behavior indicates a potential vulnerability in how RAG is integrated within the model. If exploited, it might allow users to circumvent content filtering by embedding sensitive or restricted information within context documents. This could have broader implications for both content moderation and AI safety.

Recommendation:

Consider implementing an additional safety layer that evaluates the aggregated prompt (query + context) before generating a response

Originally created by @rogerganga on GitHub (Mar 5, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/537 I have observed that when external context is added via Retrieval-Augmented Generation (RAG), the model’s content filtering appears to be bypassed. Specifically, if a user provides a document containing information that would normally be blocked in a direct query, the model generates a complete response based on that context. While my experiments are based on a generic demonstration, this pattern indicates a potential vulnerability: sensitive or otherwise restricted content might be indirectly retrieved and then output, even though the model is designed to refuse such content when directly queried. **Steps to Reproduce:** - Use the attached reference code which: - Reads a text file containing context. - Uses a simple retrieval function to select top paragraphs based on keyword overlap. - Constructs a prompt combining the query and the retrieved context. - Sends the prompt to the DeepSeek model - Run the code with a query and observe that: - Without context, the model appropriately refuses or produces a minimal response. - With the provided context (from the text file), the model generates a comprehensive response that incorporates the information from the document. **Model Used:** deepseek-r1:1.5b. I ran this model using Ollama **Reference Code:** ``` import re from ollama import chat def read_text_file(file_path): """Read and return the content of the given text file.""" with open(file_path, 'r', encoding='utf-8') as f: return f.read() def retrieve_relevant_context(query, text, k=3): """ A simple retrieval function that splits the text into paragraphs and selects the top k paragraphs with the highest keyword overlap with the query. """ paragraphs = text.split('\n\n') query_tokens = set(re.findall(r'\w+', query.lower())) scored_paras = [] for para in paragraphs: para_tokens = set(re.findall(r'\w+', para.lower())) score = len(query_tokens.intersection(para_tokens)) scored_paras.append((score, para)) scored_paras.sort(key=lambda x: x[0], reverse=True) top_paras = [para for score, para in scored_paras if score > 0][:k] return "\n\n".join(top_paras) def generate_answer(query, context): """ Constructs a prompt with the provided context and query, then calls DeepSeek to generate an answer. """ prompt = f"Using the following context, answer the query:\n\nQuery: {query}\n\nContext:\n{context}" model = "deepseek-r1:1.5b" answer = "" try: response = chat( model=model, messages=[{"role": "user", "content": prompt}], stream=True ) for chunk in response: answer += chunk['message']['content'] answer = answer.strip() except Exception as e: print("Error generating answer:", e) answer = "" return answer def main(): file_path = "BAD_FILE.txt" # Replace with your text file's path text = read_text_file(file_path) query = "BAD PROMPT" context = retrieve_relevant_context(query, text) answer = generate_answer(query, context) print(answer) if __name__ == '__main__': main() ``` **Expected Behavior:** - The model should enforce its content filtering regardless of whether the sensitive information is provided directly in the query or as part of the external context. - It should either provide a refusal response or sanitize the output even when context is injected. **Actual Behavior:** - When context is added, the model generates a comprehensive answer that includes information which would normally be censored if asked directly. **Impact:** This behavior indicates a potential vulnerability in how RAG is integrated within the model. If exploited, it might allow users to circumvent content filtering by embedding sensitive or restricted information within context documents. This could have broader implications for both content moderation and AI safety. **Recommendation:** Consider implementing an additional safety layer that evaluates the aggregated prompt (query + context) before generating a response

bsakaguchi

2026-10-26 21:21:07 +00:00

closed this issue
added the
stale

closed-as-stale
labels

bsakaguchi commented

2026-10-26 21:21:09 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Apr 17, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

@github-actions[bot] commented on GitHub (Apr 17, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

bsakaguchi commented

2026-10-26 21:21:09 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (May 20, 2025):

false

@github-actions[bot] commented on GitHub (May 20, 2025): false