[GH-ISSUE #88] When using the deepseek or generating content, the output contains the character � instead of expected characters. #74

New issue

Closed

opened 2026-10-26 21:03:06 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 21:03:06 +00:00

Owner

Originally created by @BinhMinhs10 on GitHub (Dec 29, 2023).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/88

Details:
Upon generating content with deepseek, the output includes the special character �, which is not the intended behavior. This issue might impact the readability and usability of the generated content.

Expected Behavior:
The generated content should use appropriate characters and avoid the presence of � in the output

Screenshots:

Steps to Reproduce:
Use the deepseek coder generation process.
For example, input the phrase "Bạn có thể làm gì" .
Examine the generated content.

Originally created by @BinhMinhs10 on GitHub (Dec 29, 2023). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/88 **Details:** Upon generating content with deepseek, the output includes the special character �, which is not the intended behavior. This issue might impact the readability and usability of the generated content. **Expected Behavior:** The generated content should use appropriate characters and avoid the presence of � in the output **Screenshots:** ![Screenshot from 2023-12-29 09-59-00](https://github.com/deepseek-ai/DeepSeek-Coder/assets/22696815/6c7a557f-7ea1-469d-84e1-078d852f130e) **Steps to Reproduce:** Use the deepseek coder generation process. For example, input the phrase "Bạn có thể làm gì" . Examine the generated content.

bsakaguchi closed this issue

2026-10-26 21:03:06 +00:00

bsakaguchi commented

2026-10-26 21:03:08 +00:00

Author

Owner

@DOGEwbx commented on GitHub (Jan 2, 2024):

Hello @BinhMinhs10, I believe the letter that should appear in the � place must be ú. This is a decoding mistake originating from Huggingface tokenizers. You can find more information at this link: https://github.com/deepseek-ai/DeepSeek-LLM/issues/9#issuecomment-1835679866

@DOGEwbx commented on GitHub (Jan 2, 2024): Hello @BinhMinhs10, I believe the letter that should appear in the � place must be ú. This is a decoding mistake originating from Huggingface tokenizers. You can find more information at this link: https://github.com/deepseek-ai/DeepSeek-LLM/issues/9#issuecomment-1835679866

bsakaguchi commented

2026-10-26 21:03:08 +00:00

Author

Owner

@BinhMinhs10 commented on GitHub (Jan 2, 2024):

@DOGEwbx Thank you, and I really looking forward to your update!

@BinhMinhs10 commented on GitHub (Jan 2, 2024): @DOGEwbx Thank you, and I really looking forward to your update!