[GH-ISSUE #3713] [Feature] XNB Content Pipeline reverse engineering: XNB type reader descriptors, compressed textures, sprite fonts, audio streams and 3D mesh assets #1424

New issue

Open

opened 2026-10-16 21:05:19 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:05:19 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3713

Problem Statement & Context

The XNA and MonoGame Content Pipeline compiles source art assets (textures, 3D models, sprite fonts, audio, particle effects) into optimized binary container files (.xnb) intended for rapid runtime loading via ContentManager.Load<T>().

While REA can inspect managed .NET CIL assemblies (#1558, #3607), game content remains opaque inside compiled .xnb files:

Container Headers & Compression:
- Magic identifier: XNB (0x58, 0x4E, 0x42).
- Target platform indicator: w (Windows), m (Windows Phone), x (Xbox 360), i (iOS), a (Android).
- Format version: 4 (XNA 3.1) or 5 (XNA 4.0 / MonoGame).
- Flags: Bit 0x80 indicating LZX compression (Windows/Xbox) or LZ4 compression (modern MonoGame).
Type Reader Registry Table:
- Sequential array of fully-qualified .NET type reader assembly strings (Microsoft.Xna.Framework.Content.Texture2DReader, Microsoft.Xna.Framework.Graphics, ...).
- Type reader versions and shared resource counters.
Primary Asset Payloads:
- Texture2D: Surface format (Color, Dxt1, Dxt3, Dxt5), dimensions, mipmap count, and raw pixel blocks.
- SpriteFont: Texture sheet, glyph character mappings, kerning tables, line spacing, and default character.
- Model: Bone hierarchies, meshes, mesh parts, index buffers, vertex buffers, and basic effect shader uniforms.
- SoundEffect: WAV RIFF byte payloads, loop points, and sample rates.
Custom ContentTypeReaders: Game-specific data classes serialized via custom reader/writer implementations declared in managed assemblies.

REA needs a dedicated static .xnb parser capable of decompressing LZX/LZ4 streams, deserializing standard type readers, and recovering source assets.

Proposed Architecture

flowchart TD
    A["Content File (.xnb)"] --> B["XNB Content Pipeline Engine"]
    B --> C["XNB Header & Compression Unpacker"]
    B --> D["Type Reader Registry Deserializer"]
    B --> E["Standard & Custom Asset Recoverer"]
    
    C --> C1["Magic Verification ('XNB', Platform Flag 'w'/'x'/'a')"]
    C --> C2["Decompression (LZX Window Slicing / LZ4)"]
    
    D --> D1["Type Reader String Manifest & Assembly Signatures"]
    D --> D2["Shared Resource Pool Resolution"]
    
    E --> E1["Texture2D Extractor (DXT / RGBA -> PNG)"]
    E --> E2["SpriteFont Glyph Metrics & Raster Sheet"]
    E --> E3["Model & Mesh Vertex Buffer Slicer"]
    E --> E4["SoundEffect Audio Extractor (WAV / OGG)"]
    
    C & D & E --> F["Synthesized Asset Manifest & Extracted Multi-Format Media"]

Technical Capabilities & Modules

Header Identification & Decompression:
- Parse XNB header: verify XNB magic, target platform flag, and format version (v4 / v5).
- Decompress LZX-compressed payloads (using 32 KB sliding window) and MonoGame LZ4-compressed payloads without executing .NET runtime binaries.
Type Reader Table Parsing:
- Parse the 7-bit variable-length encoded type reader count and reader manifest strings.
- Map type reader indices to standard built-in readers: Texture2DReader, SpriteFontReader, ModelReader, SoundEffectReader, EffectReader, DictionaryReader, ListReader.
Standard Asset Deserialization & Conversion:
- Textures (Texture2D): Decode Color (RGBA), DXT1, DXT3, and DXT5 surfaces; export mipmaps to standard PNG images.
- Fonts (SpriteFont): Extract font atlas textures and generate character map JSON files with kerning pairs and spacing metrics.
- Audio (SoundEffect): Extract RIFF WAV audio streams with duration, loop start, and loop length parameters.
- Meshes (Model): Reconstruct skeletal bones, vertex position buffers, normals, texture coordinates, and triangle index lists.
Custom ContentTypeReader Correlation:
- Identify custom user-defined type reader strings in the XNB header.
- Correlate reader class names with decompiled managed assemblies from the accompanying application executable to map custom field deserialization logic.

Acceptance Criteria

Automated detection and decompression of LZX and LZ4 compressed .xnb files.
Parsing of the type reader registry table across XNA 4.0 and MonoGame versions.
Clean extraction and conversion of Texture2D assets to standard PNG format.
Extraction of SpriteFont glyph metrics and SoundEffect audio streams.
Export of an asset summary report detailing payload types, dimensions, and type reader signatures.

Complements #3621 (Unknown binary file-format inference)
Complements #3607 (Comprehensive Unity extraction: Mono CIL)
Complements #3698 (Interactive entertainment runtime and asset reverse engineering tracker)

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3713 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ### Problem Statement & Context The XNA and MonoGame Content Pipeline compiles source art assets (textures, 3D models, sprite fonts, audio, particle effects) into optimized binary container files (`.xnb`) intended for rapid runtime loading via `ContentManager.Load<T>()`. While REA can inspect managed .NET CIL assemblies (#1558, #3607), game content remains opaque inside compiled `.xnb` files: 1. **Container Headers & Compression**: - Magic identifier: `XNB` (`0x58`, `0x4E`, `0x42`). - Target platform indicator: `w` (Windows), `m` (Windows Phone), `x` (Xbox 360), `i` (iOS), `a` (Android). - Format version: 4 (XNA 3.1) or 5 (XNA 4.0 / MonoGame). - Flags: Bit 0x80 indicating LZX compression (Windows/Xbox) or LZ4 compression (modern MonoGame). 2. **Type Reader Registry Table**: - Sequential array of fully-qualified .NET type reader assembly strings (`Microsoft.Xna.Framework.Content.Texture2DReader, Microsoft.Xna.Framework.Graphics, ...`). - Type reader versions and shared resource counters. 3. **Primary Asset Payloads**: - `Texture2D`: Surface format (Color, Dxt1, Dxt3, Dxt5), dimensions, mipmap count, and raw pixel blocks. - `SpriteFont`: Texture sheet, glyph character mappings, kerning tables, line spacing, and default character. - `Model`: Bone hierarchies, meshes, mesh parts, index buffers, vertex buffers, and basic effect shader uniforms. - `SoundEffect`: WAV RIFF byte payloads, loop points, and sample rates. 4. **Custom ContentTypeReaders**: Game-specific data classes serialized via custom reader/writer implementations declared in managed assemblies. REA needs a dedicated static `.xnb` parser capable of decompressing LZX/LZ4 streams, deserializing standard type readers, and recovering source assets. ### Proposed Architecture ```mermaid flowchart TD A["Content File (.xnb)"] --> B["XNB Content Pipeline Engine"] B --> C["XNB Header & Compression Unpacker"] B --> D["Type Reader Registry Deserializer"] B --> E["Standard & Custom Asset Recoverer"] C --> C1["Magic Verification ('XNB', Platform Flag 'w'/'x'/'a')"] C --> C2["Decompression (LZX Window Slicing / LZ4)"] D --> D1["Type Reader String Manifest & Assembly Signatures"] D --> D2["Shared Resource Pool Resolution"] E --> E1["Texture2D Extractor (DXT / RGBA -> PNG)"] E --> E2["SpriteFont Glyph Metrics & Raster Sheet"] E --> E3["Model & Mesh Vertex Buffer Slicer"] E --> E4["SoundEffect Audio Extractor (WAV / OGG)"] C & D & E --> F["Synthesized Asset Manifest & Extracted Multi-Format Media"] ``` ### Technical Capabilities & Modules 1. **Header Identification & Decompression:** - Parse XNB header: verify `XNB` magic, target platform flag, and format version (v4 / v5). - Decompress LZX-compressed payloads (using 32 KB sliding window) and MonoGame LZ4-compressed payloads without executing .NET runtime binaries. 2. **Type Reader Table Parsing:** - Parse the 7-bit variable-length encoded type reader count and reader manifest strings. - Map type reader indices to standard built-in readers: `Texture2DReader`, `SpriteFontReader`, `ModelReader`, `SoundEffectReader`, `EffectReader`, `DictionaryReader`, `ListReader`. 3. **Standard Asset Deserialization & Conversion:** - **Textures (`Texture2D`)**: Decode Color (RGBA), DXT1, DXT3, and DXT5 surfaces; export mipmaps to standard PNG images. - **Fonts (`SpriteFont`)**: Extract font atlas textures and generate character map JSON files with kerning pairs and spacing metrics. - **Audio (`SoundEffect`)**: Extract RIFF WAV audio streams with duration, loop start, and loop length parameters. - **Meshes (`Model`)**: Reconstruct skeletal bones, vertex position buffers, normals, texture coordinates, and triangle index lists. 4. **Custom ContentTypeReader Correlation:** - Identify custom user-defined type reader strings in the XNB header. - Correlate reader class names with decompiled managed assemblies from the accompanying application executable to map custom field deserialization logic. ### Acceptance Criteria - [ ] Automated detection and decompression of LZX and LZ4 compressed `.xnb` files. - [ ] Parsing of the type reader registry table across XNA 4.0 and MonoGame versions. - [ ] Clean extraction and conversion of `Texture2D` assets to standard PNG format. - [ ] Extraction of `SpriteFont` glyph metrics and `SoundEffect` audio streams. - [ ] Export of an asset summary report detailing payload types, dimensions, and type reader signatures. ### Related Issues & Dependencies - Complements #3621 (Unknown binary file-format inference) - Complements #3607 (Comprehensive Unity extraction: Mono CIL) - Complements #3698 (Interactive entertainment runtime and asset reverse engineering tracker)

bsakaguchi referenced this issue

2026-10-17 21:01:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:11:18 +00:00

[PR #1424] [MERGED] fix(javascript): unify package resolution and resolve exact exports subpaths #2876

bsakaguchi referenced this issue

2026-10-31 00:54:09 +00:00

[PR #1615] [MERGED] fix(javascript): preserve trailing slash in package directory requests #4701