[GH-ISSUE #461] PGVector gotcha #70

New issue

Closed

opened 2026-10-18 14:08:41 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-18 14:08:41 +00:00

Owner

Originally created by @humbertovnavarro on GitHub (Mar 28, 2025).
Original GitHub issue: https://github.com/open-webui/docs/issues/461

By default postgres uses ASCII encoding which causes document uploads to fail if they contain unicode characters.

Originally created by @humbertovnavarro on GitHub (Mar 28, 2025). Original GitHub issue: https://github.com/open-webui/docs/issues/461 By default postgres uses ASCII encoding which causes document uploads to fail if they contain unicode characters.

bsakaguchi

2026-10-18 14:08:41 +00:00

closed this issue
added the
help wanted

documentation
labels

bsakaguchi commented

2026-10-18 14:08:42 +00:00

Author

Owner

@mihirahlawat commented on GitHub (Apr 17, 2025):

I spent a lot of time and will try to save you some.

The issue: If PDFs have ligatures in them, PyPDF does not recognize them and gives you an empty character (in unicode) which throws error in a PGVector Database.

The solution: I used PyMuPDF for loading pdf documents. It is able to recognize ligatures seamlessly.

@mihirahlawat commented on GitHub (Apr 17, 2025): I spent a lot of time and will try to save you some. The issue: If PDFs have ligatures in them, PyPDF does not recognize them and gives you an empty character (in unicode) which throws error in a PGVector Database. The solution: I used PyMuPDF for loading pdf documents. It is able to recognize ligatures seamlessly.

bsakaguchi commented

2026-10-18 14:08:42 +00:00

Author

Owner

@Classic298 commented on GitHub (Sep 14, 2025):

Hi, what improvement to the docs do you propose based on these observations? Not sure where to add it?

@Classic298 commented on GitHub (Sep 14, 2025): Hi, what improvement to the docs do you propose based on these observations? Not sure where to add it?

bsakaguchi referenced this issue

2026-10-18 14:10:19 +00:00

[PR #70] [CLOSED] added Dalle3 image generation tutorial #221