[PR #4715] [CLOSED] fix(ghidra): preserve valid replacement characters in language catalog #6367

New issue

Closed

opened 2026-10-31 01:13:55 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:13:55 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4715
Author: @yadavd4631
Created: 10/17/2026
Status: ❌ Closed

Base: main ← Head: fix-ghidra-ldefs-replacement-chars

📝 Commits (1)

21be1c1 fix(ghidra): preserve valid replacement characters in language catalog

📊 Changes

2 files changed (+138 additions, -2 deletions)

View changed files

➕ src/ghidra/GhidraLanguageCatalog.test.ts (+125 -0)
📝 src/ghidra/GhidraLanguageCatalog.ts (+13 -2)

📄 Description

Summary

Fixes #4712

Allow Ghidra language catalog (.ldefs) parsing to succeed when descriptions contain the legal Unicode replacement character \uFFFD, and decode catalog files using a fatal UTF-8 TextDecoder so malformed bytes are rejected before XML parsing.

Problem and expected behavior

Previously, parseGhidraLanguageDefinitions forwarded all warnings from @xmldom/xmldom directly to onWarningStopParsing. xmldom emits an advisory warning ("Unicode replacement character detected, source encoding issues?") whenever \uFFFD is present in the document. Because onWarningStopParsing unconditionally throws on warning, any valid .ldefs file containing \uFFFD in an unrelated description field failed provider discovery and setup. Furthermore, readGhidraLanguageCatalog used a non-fatal TextDecoder, which masked actual byte corruption into \uFFFD rather than throwing a fatal byte decoding error.

With this change:

In parseGhidraLanguageDefinitions, the diagnostic warning for Unicode replacement characters is ignored while all other warnings/errors continue to be rejected by onWarningStopParsing().
In readGhidraLanguageCatalog, TextDecoder("utf-8", { fatal: true }) ensures invalid UTF-8 bytes throw a TypeError before reaching the XML parser.
Valid language and compiler definitions are preserved.

Validation

Added unit tests in src/ghidra/GhidraLanguageCatalog.test.ts covering:
- Preserving definitions when description contains literal \uFFFD
- Continuing to reject malformed XML syntax containing \uFFFD
- Rejection of invalid UTF-8 byte sequences with TypeError
- Stripping UTF-8 BOM before XML parsing
Ran existing test suites:
- src/ghidra/GhidraLanguageOverride.test.ts (20 passed)
- tests/boundary/filesystem/ghidraLanguageCatalogBound.test.ts (2 passed)
- tests/boundary/filesystem/ghidraLanguageCatalogCleanup.test.ts (5 passed)
- src/ghidra/GhidraLanguageCatalog.test.ts (7 passed)
- Total: 34 tests passed cleanly.
npm run check:fast (typecheck + lint) passed with 0 errors.
oxfmt --check passed cleanly across modified files.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4715 **Author:** [@yadavd4631](https://github.com/yadavd4631) **Created:** 10/17/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix-ghidra-ldefs-replacement-chars` --- ### 📝 Commits (1) - [`21be1c1`](https://github.com/morluto/rea/commit/21be1c195f2106e2eb1287dbd5eaee91530f7361) fix(ghidra): preserve valid replacement characters in language catalog ### 📊 Changes **2 files changed** (+138 additions, -2 deletions) <details> <summary>View changed files</summary> ➕ `src/ghidra/GhidraLanguageCatalog.test.ts` (+125 -0) 📝 `src/ghidra/GhidraLanguageCatalog.ts` (+13 -2) </details> ### 📄 Description ## Summary Fixes #4712 Allow Ghidra language catalog (.ldefs) parsing to succeed when descriptions contain the legal Unicode replacement character `\uFFFD`, and decode catalog files using a fatal UTF-8 TextDecoder so malformed bytes are rejected before XML parsing. ## Problem and expected behavior Previously, `parseGhidraLanguageDefinitions` forwarded all warnings from `@xmldom/xmldom` directly to `onWarningStopParsing`. xmldom emits an advisory warning (`"Unicode replacement character detected, source encoding issues?"`) whenever `\uFFFD` is present in the document. Because `onWarningStopParsing` unconditionally throws on warning, any valid `.ldefs` file containing `\uFFFD` in an unrelated description field failed provider discovery and setup. Furthermore, `readGhidraLanguageCatalog` used a non-fatal `TextDecoder`, which masked actual byte corruption into `\uFFFD` rather than throwing a fatal byte decoding error. With this change: - In `parseGhidraLanguageDefinitions`, the diagnostic warning for Unicode replacement characters is ignored while all other warnings/errors continue to be rejected by `onWarningStopParsing()`. - In `readGhidraLanguageCatalog`, `TextDecoder("utf-8", { fatal: true })` ensures invalid UTF-8 bytes throw a `TypeError` before reaching the XML parser. - Valid language and compiler definitions are preserved. ## Validation - Added unit tests in `src/ghidra/GhidraLanguageCatalog.test.ts` covering: - Preserving definitions when description contains literal `\uFFFD` - Continuing to reject malformed XML syntax containing `\uFFFD` - Rejection of invalid UTF-8 byte sequences with `TypeError` - Stripping UTF-8 BOM before XML parsing - Ran existing test suites: - `src/ghidra/GhidraLanguageOverride.test.ts` (20 passed) - `tests/boundary/filesystem/ghidraLanguageCatalogBound.test.ts` (2 passed) - `tests/boundary/filesystem/ghidraLanguageCatalogCleanup.test.ts` (5 passed) - `src/ghidra/GhidraLanguageCatalog.test.ts` (7 passed) - Total: 34 tests passed cleanly. - `npm run check:fast` (typecheck + lint) passed with 0 errors. - `oxfmt --check` passed cleanly across modified files. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:13:55 +00:00