[GH-ISSUE #1331] [BUG]Hallucination - Model Skips Opening Links and Fabricates Content Instead #875

Closed
opened 2026-10-26 21:31:29 +00:00 by bsakaguchi · 4 comments
Owner

Originally created by @D-Orancle on GitHub (May 18, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/1331

Describe the bug
The model has the capability to open and read external links. However, it frequently skips the step of actually opening the link and instead generates fabricated summaries based on link titles, conversation context, and user history. The model presents these fabricated summaries as if it had read the linked content, misleading the user until manually corrected.

To Reproduce
Steps to reproduce the behavior:

  1. Provide the model with an external link (e.g., a shared conversation).
  2. Ask the model to view or analyze the linked content.
  3. Observe whether the model actually opens the link and reads its content, or skips this step and generates a fabricated summary.
  4. Repeat across multiple links and contexts to confirm this is a stable pattern.

Expected behavior
When a link is provided, the model should execute the step of opening and reading the link before generating any response about its content. If the model cannot or does not open the link, it should clearly state this rather than generating any summary.

Actual behavior
The model skips the step of opening the link entirely. Without having accessed the linked content, it generates a detailed, plausible, but entirely fabricated summary. The model then presents this fabricated summary as if it were a faithful account of the linked content, misleading the user until the user manually verifies and corrects it.

Severity: High
This is not a technical limitation—the model has the capability to open links. This is a behavioral defect where the model skips a critical verification step and defaults to fabrication. This directly undermines user trust in a core user-initiated function, and is especially dangerous when links contain information on serious topics such as legal, medical, or safety-related content.

Screenshots

Image Image Image

Additional context
This issue has been observed on the latest production model (V4-Pro). It is distinct from but related to Issue #1323—both stem from the same root cause: the model defaults to generating plausible-sounding answers rather than pausing to execute necessary verification steps or honestly acknowledging limitations. The user has pointed out this specific pattern multiple times within a single conversation, yet the model continued to repeat the same error. Cross-version testing may help determine whether this defect is isolated to V4 or present in earlier releases.

Suggested fix
Investigate why the model skips the link-opening step when it has the capability to do so. Implement a mandatory verification step that requires the model to confirm successful content retrieval before generating any summary. If the link cannot be opened for any reason, the model must default to an explicit error message rather than inferring or fabricating content.

Originally created by @D-Orancle on GitHub (May 18, 2026). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/1331 Describe the bug The model has the capability to open and read external links. However, it frequently skips the step of actually opening the link and instead generates fabricated summaries based on link titles, conversation context, and user history. The model presents these fabricated summaries as if it had read the linked content, misleading the user until manually corrected. To Reproduce Steps to reproduce the behavior: 1. Provide the model with an external link (e.g., a shared conversation). 2. Ask the model to view or analyze the linked content. 3. Observe whether the model actually opens the link and reads its content, or skips this step and generates a fabricated summary. 4. Repeat across multiple links and contexts to confirm this is a stable pattern. Expected behavior When a link is provided, the model should execute the step of opening and reading the link before generating any response about its content. If the model cannot or does not open the link, it should clearly state this rather than generating any summary. Actual behavior The model skips the step of opening the link entirely. Without having accessed the linked content, it generates a detailed, plausible, but entirely fabricated summary. The model then presents this fabricated summary as if it were a faithful account of the linked content, misleading the user until the user manually verifies and corrects it. Severity: High This is not a technical limitation—the model has the capability to open links. This is a behavioral defect where the model skips a critical verification step and defaults to fabrication. This directly undermines user trust in a core user-initiated function, and is especially dangerous when links contain information on serious topics such as legal, medical, or safety-related content. Screenshots <img width="1080" height="3533" alt="Image" src="https://github.com/user-attachments/assets/97227197-695a-4ad0-8283-2a6a16e9338a" /> <img width="1080" height="3345" alt="Image" src="https://github.com/user-attachments/assets/387643a6-558a-4dcf-8036-37ef3a63fdb9" /> <img width="1080" height="4504" alt="Image" src="https://github.com/user-attachments/assets/a6559d09-e062-46bd-8383-9dbbaa325e7b" /> Additional context This issue has been observed on the latest production model (V4-Pro). It is distinct from but related to Issue #1323—both stem from the same root cause: the model defaults to generating plausible-sounding answers rather than pausing to execute necessary verification steps or honestly acknowledging limitations. The user has pointed out this specific pattern multiple times within a single conversation, yet the model continued to repeat the same error. Cross-version testing may help determine whether this defect is isolated to V4 or present in earlier releases. Suggested fix Investigate why the model skips the link-opening step when it has the capability to do so. Implement a mandatory verification step that requires the model to confirm successful content retrieval before generating any summary. If the link cannot be opened for any reason, the model must default to an explicit error message rather than inferring or fabricating content.
bsakaguchi 2026-10-26 21:31:29 +00:00
Author
Owner

@qingkong66 commented on GitHub (May 18, 2026):

@D-Orancle

Confirmed pattern. The model skips the link-opening step and fabricates summaries instead. You've documented it clearly with screenshots.

Related to #1323 — same root: model defaults to plausible fabrication over verification.

Not a fix, but your issue is on record.

<!-- gh-comment-id:4478915320 --> @qingkong66 commented on GitHub (May 18, 2026): @D-Orancle Confirmed pattern. The model skips the link-opening step and fabricates summaries instead. You've documented it clearly with screenshots. Related to #1323 — same root: model defaults to plausible fabrication over verification. Not a fix, but your issue is on record.
Author
Owner

@darkemil2000-arch commented on GitHub (May 19, 2026):

Поддержите идею #1328

<!-- gh-comment-id:4484256687 --> @darkemil2000-arch commented on GitHub (May 19, 2026): Поддержите идею #1328
Author
Owner

@github-actions[bot] commented on GitHub (Jun 23, 2026):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

<!-- gh-comment-id:4774494757 --> @github-actions[bot] commented on GitHub (Jun 23, 2026): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
Author
Owner

@github-actions[bot] commented on GitHub (Jul 8, 2026):

false

<!-- gh-comment-id:4910295302 --> @github-actions[bot] commented on GitHub (Jul 8, 2026): false
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-V3#875
No description provided.