[GH-ISSUE #2531] fix(ghidra): oversized language files allocate beyond the catalog limit #849

New issue

Closed

opened 2026-10-15 18:28:58 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:28:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2531

Summary

When REA checks a configured Ghidra language override, an installation language file larger than the catalog's 4 MiB limit still becomes a full memory buffer before REA skips it. The limit therefore fails to bound this allocation. A focused probe observed an 8 MiB file buffered in full while a small valid sibling remained usable.

Runtime or environment

Repository revision 8b181716ac, Node v26.10.0, npm 11.19.1, and locked dependencies. Verification exercised the production catalog reader with real filesystem fixtures; it did not launch Ghidra.

Steps to reproduce

In a disposable installation root, create Ghidra/Processors/probe/data/languages/.
Write a-big.ldefs as <language_definitions> followed by 8,388,608 spaces and <language id="big"/></language_definitions>. Write ok.ldefs as <language_definitions><language id="ok"/></language_definitions>.
Call readGhidraLanguageCatalog on that root, observing the lengths of buffers returned by the real filesystem reads before the reader processes them. This is the same catalog operation used when an available Ghidra provider admits a configured language override after resolving a supported target profile.
Compare the allocated buffer lengths with the 4 MiB limit and inspect the returned catalog. The focused local probe executed these steps once.

Expected behavior

REA should refuse oversized language definitions without allocating their full contents. It should still return language definitions from eligible sibling files, preserving the existing resource limit and catalog behavior.

Actual behavior

The real filesystem reads returned buffers of 8,388,673 and 64 bytes. The catalog returned only ok, with no compiler specs. Filtering worked, but the oversized file had already consumed a buffer roughly twice the stated limit. Process exhaustion was not tested.

Evidence

Expected source: src/ghidra/GhidraLanguageCatalog.ts:14, the 4 MiB refusal policy, and AGENTS.md, Tool and Test Changes, which requires allocation-aware resource constraints.
Failure source: src/ghidra/GhidraLanguageCatalog.ts:76, the full read before the length guard; the focused probe observed the actual oversized buffer returned by that read. src/ghidra/GhidraProvider.ts:213 reaches the catalog for a configured override and available installation after profile resolution.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The probe used real file reads with an observation wrapper and preserved the valid sibling. No matching issue was found in read-only searches for the catalog and language files.

Restoration check

Repeat the oversized-file and valid-sibling fixture. The reader must refuse the oversized file without allocating its whole contents, and must still return ok. Also check a valid file at the admitted size boundary to establish that the bound does not suppress eligible definitions.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A malformed or modified local installation file defeats a concrete allocation bound during language-override admission. The observed allocation is consequential but no crash or broad reach was established.
Workaround status: available
Workaround basis: Restore the oversized modified file from a known-good small copy, or omit an unnecessary override to avoid the catalog admission path. This does not require weakening an authority boundary.

Originally created by @coygeek on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2531 ## Summary When REA checks a configured Ghidra language override, an installation language file larger than the catalog's 4 MiB limit still becomes a full memory buffer before REA skips it. The limit therefore fails to bound this allocation. A focused probe observed an 8 MiB file buffered in full while a small valid sibling remained usable. ## Runtime or environment Repository revision 8b181716aca2cb467be499cba226abfb5ae19141, Node v26.10.0, npm 11.19.1, and locked dependencies. Verification exercised the production catalog reader with real filesystem fixtures; it did not launch Ghidra. ## Steps to reproduce 1. In a disposable installation root, create `Ghidra/Processors/probe/data/languages/`. 2. Write `a-big.ldefs` as `<language_definitions>` followed by 8,388,608 spaces and `<language id="big"/></language_definitions>`. Write `ok.ldefs` as `<language_definitions><language id="ok"/></language_definitions>`. 3. Call `readGhidraLanguageCatalog` on that root, observing the lengths of buffers returned by the real filesystem reads before the reader processes them. This is the same catalog operation used when an available Ghidra provider admits a configured language override after resolving a supported target profile. 4. Compare the allocated buffer lengths with the 4 MiB limit and inspect the returned catalog. The focused local probe executed these steps once. ## Expected behavior REA should refuse oversized language definitions without allocating their full contents. It should still return language definitions from eligible sibling files, preserving the existing resource limit and catalog behavior. ## Actual behavior The real filesystem reads returned buffers of 8,388,673 and 64 bytes. The catalog returned only `ok`, with no compiler specs. Filtering worked, but the oversized file had already consumed a buffer roughly twice the stated limit. Process exhaustion was not tested. ## Evidence - Expected source: `src/ghidra/GhidraLanguageCatalog.ts:14`, the 4 MiB refusal policy, and `AGENTS.md`, Tool and Test Changes, which requires allocation-aware resource constraints. - Failure source: `src/ghidra/GhidraLanguageCatalog.ts:76`, the full read before the length guard; the focused probe observed the actual oversized buffer returned by that read. `src/ghidra/GhidraProvider.ts:213` reaches the catalog for a configured override and available installation after profile resolution. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The probe used real file reads with an observation wrapper and preserved the valid sibling. No matching issue was found in read-only searches for the catalog and language files. ## Restoration check Repeat the oversized-file and valid-sibling fixture. The reader must refuse the oversized file without allocating its whole contents, and must still return `ok`. Also check a valid file at the admitted size boundary to establish that the bound does not suppress eligible definitions. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A malformed or modified local installation file defeats a concrete allocation bound during language-override admission. The observed allocation is consequential but no crash or broad reach was established. - Workaround status: available - Workaround basis: Restore the oversized modified file from a known-good small copy, or omit an unnecessary override to avoid the catalog admission path. This does not require weakening an authority boundary.

bsakaguchi

2026-10-15 18:28:58 +00:00

closed this issue
added the
area: native

bug
labels

bsakaguchi commented

2026-10-15 18:28:59 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Verified the reported defect is addressed on main by cb157084aa.

Root cause and resolution: Language definition files were materialized before the catalog byte limit was enforced, allowing oversized input allocation.

Validation: tests/boundary/filesystem/ghidraLanguageCatalogBound.test.ts — skips an oversized definition without buffering it beyond the catalog limit rejects an 8 MiB file while retaining a valid sibling; exact-limit file is accepted and byte delivery is bounded.

Evidence: CI run 500e: owner unchanged and assertions passed.

Verification scope: Real filesystem fixtures, no Ghidra process.

Current main CI has separate failures under repair; this closure does not claim the complete main workflow is green.

@morluto commented on GitHub (Oct 14, 2026): Verified the reported defect is addressed on main by cb157084aab65c66192991416de732bd411d7af9. Root cause and resolution: Language definition files were materialized before the catalog byte limit was enforced, allowing oversized input allocation. Validation: `tests/boundary/filesystem/ghidraLanguageCatalogBound.test.ts` — `skips an oversized definition without buffering it beyond the catalog limit` rejects an 8 MiB file while retaining a valid sibling; exact-limit file is accepted and byte delivery is bounded. Evidence: CI run 500e: owner unchanged and assertions passed. Verification scope: Real filesystem fixtures, no Ghidra process. Current main CI has separate failures under repair; this closure does not claim the complete main workflow is green.

bsakaguchi referenced this issue

2026-10-16 21:16:46 +00:00

[PR #849] [MERGED] test: eliminate fixture races and prune redundant scaffolding #2287

bsakaguchi referenced this issue

2026-10-16 21:16:50 +00:00

[PR #854] [MERGED] fix: preserve native UI failure reasons and dedupe version literals #2290