[GH-ISSUE #152] um kb search crashes with UnicodeEncodeError on Windows (cp1252) #35

New issue

Closed

opened 2026-10-15 20:52:43 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 20:52:43 +00:00

Owner

Originally created by @olovorgo-discord on GitHub (Oct 8, 2026).
Original GitHub issue: https://github.com/rehan-remade/universal-modder/issues/152

What

um kb search <term> crashes on Windows when a matching field note contains a character outside cp1252:

File "um\kb.py", line 390, in main
    print(f"     | {h.lstrip('> ')[:160]}")
File "...\Lib\encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character ...

Repro

Windows 11, Python 3.11 (uv-managed), um 0.2.0 installed via uv tool install git+https://github.com/rehan-remade/universal-modder
um kb search aoe — crashes while printing hits from knowledge/games/age-of-empires-ii-de/san-franciscans-civ.md (that note contains non-cp1252 characters)

Root cause

Python picks the console's legacy encoding (cp1252) for stdout on Windows, and kb.py prints raw note text (hit snippets) that can contain characters cp1252 can't represent.

Workaround

$env:PYTHONUTF8 = "1" before running um kb — works fine.

Suggested fix

Reconfigure stdout to UTF-8 at startup in um/cli.py main (e.g. sys.stdout.reconfigure(encoding="utf-8") guarded by try/except), or sanitize hit text before printing. Same latent issue in any other subcommand that prints note/repo text.

Originally created by @olovorgo-discord on GitHub (Oct 8, 2026). Original GitHub issue: https://github.com/rehan-remade/universal-modder/issues/152 ## What `um kb search <term>` crashes on Windows when a matching field note contains a character outside cp1252: ``` File "um\kb.py", line 390, in main print(f" | {h.lstrip('> ')[:160]}") File "...\Lib\encodings\cp1252.py", line 19, in encode UnicodeEncodeError: 'charmap' codec can't encode character ... ``` ## Repro - Windows 11, Python 3.11 (uv-managed), `um` 0.2.0 installed via `uv tool install git+https://github.com/rehan-remade/universal-modder` - `um kb search aoe` — crashes while printing hits from `knowledge/games/age-of-empires-ii-de/san-franciscans-civ.md` (that note contains non-cp1252 characters) ## Root cause Python picks the console's legacy encoding (cp1252) for stdout on Windows, and kb.py prints raw note text (hit snippets) that can contain characters cp1252 can't represent. ## Workaround `$env:PYTHONUTF8 = "1"` before running `um kb` — works fine. ## Suggested fix Reconfigure stdout to UTF-8 at startup in `um/cli.py` main (e.g. `sys.stdout.reconfigure(encoding="utf-8")` guarded by try/except), or sanitize hit text before printing. Same latent issue in any other subcommand that prints note/repo text.

bsakaguchi closed this issue

2026-10-15 20:52:44 +00:00

bsakaguchi commented

2026-10-15 20:52:45 +00:00

Author

Owner

@rehan-remade commented on GitHub (Oct 10, 2026):

Thanks for the precise report and the workaround. #154 makes um reconfigure stdout and stderr to UTF-8 at startup, which is the fix you suggested, and adds a test that runs um kb show under PYTHONIOENCODING=cp1252. This closes once that merges. Until then, $env:PYTHONUTF8 = "1" is the workaround.

@rehan-remade commented on GitHub (Oct 10, 2026): Thanks for the precise report and the workaround. #154 makes `um` reconfigure stdout and stderr to UTF-8 at startup, which is the fix you suggested, and adds a test that runs `um kb show` under `PYTHONIOENCODING=cp1252`. This closes once that merges. Until then, `$env:PYTHONUTF8 = "1"` is the workaround.

bsakaguchi referenced this issue

2026-10-15 20:53:12 +00:00

[PR #35] [MERGED] kb: remove the Stick It To The Stickman note #74