[GH-ISSUE #3709] [Feature] KPKA container and RSZ serialization reverse engineering: KPKA archives, RSZ object reflection graphs, PFB prefabs, SCN scenes and texture dictionaries #1422

New issue

Open

opened 2026-10-16 21:05:18 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:05:18 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3709

Problem Statement & Context

Modern AAA photorealistic game architectures leverage a unified asset packaging and in-engine serialization framework centered around:

KPKA Container Archives (.pak with magic KPKA): Multi-file packages containing 16-byte header signatures (KPKA), archive versions, entry counts, and file tables. Supports Deflate, Zstandard, and Oodle Kraken compression along with optional AES encryption layers.
Runtime Serialization Format (RSZ): In-engine object graph reflection format (RSZ magic header). Serializes strongly-typed C++ class hierarchies, object fields, nested child instances, component arrays, and polymorphic pointers mapped via 32-bit MurmurHash3 / CRC type hashes.
Scenes & Prefabs (.scn, .pfb): Hierarchical scene node trees (GameObject, Transform, Folder), game components, resource references, and component overrides.
Mesh & Material Buffers (.mesh, .mdf2): Multi-LOD skeletal vertex buffers, bone weight matrices, blendshapes, and multi-pass PBR material definition files.
Texture Containers (.tex): Specialized image archives packaging mipmapped textures (BC1, BC5, BC7, ASTC) and cubemaps.

REA lacks a native parser for KPKA archive containers, the polymorphic RSZ reflection system, and prefab component graphs.

Proposed Architecture

flowchart TD
    A["Archive Container (.pak / KPKA) / Resource (.scn, .pfb, .user)"] --> B["KPKA & RSZ Semantic Engine"]
    B --> C["KPKA Archive Slicer & Unpacker"]
    B --> D["RSZ Reflection & Object Graph Parser"]
    B --> E["Scene & Prefab Component Inspector"]
    
    C --> C1["KPKA Header Magic Verification & Entry Table"]
    C --> C2["Decompression (Zstd / Oodle / Deflate)"]
    
    D --> D1["RSZ Header & Class Hash Table Indexing"]
    D --> D2["Polymorphic Object Deserialization & Field Types"]
    D --> D3["UserData / Resource Path Linking"]
    
    E --> E1[".scn Scene Hierarchy & GameObject Tree"]
    E --> E2[".pfb Prefab Component Mappings"]
    E --> E3[".mdf2 Material & .tex Surface Extraction"]
    
    C & D & E --> F["Synthesized Scene Hierarchy, Component Dependency Graph & Asset Manifest"]

Technical Capabilities & Modules

KPKA Archive Parsing & Extraction:
- Detect KPKA magic signature and parse archive version numbers.
- Parse file entry tables: 64-bit file path hashes, offset pointers, compressed sizes, and uncompressed sizes.
- Decompress payload data using Deflate, Zstandard, and Oodle algorithms.
RSZ Reflection System Deserialization:
- Parse RSZ data blocks: magic header, object count, instance table, and element data blocks.
- Map 32-bit type hashes to human-readable class definitions and field types (int, float, string, boolean, enum, object references).
- Reconstruct cyclic and polymorphic object graphs without native engine binaries.
Scene & Prefab Reconstruction (.scn, .pfb):
- Traverse GameObject hierarchies, parent-child folder structures, and active components (e.g., renderers, colliders, audio sources, script controllers).
- Reconstruct prefab component overrides and property variations.
Material & Texture Extraction:
- Parse .mdf2 material files: extract shader property uniforms, texture slot bindings (albedo, normal, roughness, metallic, ambient occlusion).
- Decode .tex texture files: convert BC1, BC4, BC5, and BC7 compressed surfaces to PNG/DDS.

Acceptance Criteria

Parsing and safe file extraction of KPKA-formatted archive containers.
Deserialization of RSZ binary data into structured, typed JSON object graphs.
Traversal of .scn and .pfb hierarchical component trees.
Extraction and decoding of .tex textures and .mdf2 material definitions.
Export of an end-to-end scene and component relationship diagram.

Complements #3621 (Unknown binary file-format inference)
Complements #3624 (Native C++ semantic reconstruction: RTTI, vtables)
Complements #3698 (Interactive entertainment runtime and asset reverse engineering tracker)

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3709 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ### Problem Statement & Context Modern AAA photorealistic game architectures leverage a unified asset packaging and in-engine serialization framework centered around: 1. **KPKA Container Archives (`.pak` with magic `KPKA`)**: Multi-file packages containing 16-byte header signatures (`KPKA`), archive versions, entry counts, and file tables. Supports Deflate, Zstandard, and Oodle Kraken compression along with optional AES encryption layers. 2. **Runtime Serialization Format (RSZ)**: In-engine object graph reflection format (`RSZ` magic header). Serializes strongly-typed C++ class hierarchies, object fields, nested child instances, component arrays, and polymorphic pointers mapped via 32-bit MurmurHash3 / CRC type hashes. 3. **Scenes & Prefabs (`.scn`, `.pfb`)**: Hierarchical scene node trees (`GameObject`, `Transform`, `Folder`), game components, resource references, and component overrides. 4. **Mesh & Material Buffers (`.mesh`, `.mdf2`)**: Multi-LOD skeletal vertex buffers, bone weight matrices, blendshapes, and multi-pass PBR material definition files. 5. **Texture Containers (`.tex`)**: Specialized image archives packaging mipmapped textures (BC1, BC5, BC7, ASTC) and cubemaps. REA lacks a native parser for `KPKA` archive containers, the polymorphic `RSZ` reflection system, and prefab component graphs. ### Proposed Architecture ```mermaid flowchart TD A["Archive Container (.pak / KPKA) / Resource (.scn, .pfb, .user)"] --> B["KPKA & RSZ Semantic Engine"] B --> C["KPKA Archive Slicer & Unpacker"] B --> D["RSZ Reflection & Object Graph Parser"] B --> E["Scene & Prefab Component Inspector"] C --> C1["KPKA Header Magic Verification & Entry Table"] C --> C2["Decompression (Zstd / Oodle / Deflate)"] D --> D1["RSZ Header & Class Hash Table Indexing"] D --> D2["Polymorphic Object Deserialization & Field Types"] D --> D3["UserData / Resource Path Linking"] E --> E1[".scn Scene Hierarchy & GameObject Tree"] E --> E2[".pfb Prefab Component Mappings"] E --> E3[".mdf2 Material & .tex Surface Extraction"] C & D & E --> F["Synthesized Scene Hierarchy, Component Dependency Graph & Asset Manifest"] ``` ### Technical Capabilities & Modules 1. **KPKA Archive Parsing & Extraction:** - Detect `KPKA` magic signature and parse archive version numbers. - Parse file entry tables: 64-bit file path hashes, offset pointers, compressed sizes, and uncompressed sizes. - Decompress payload data using Deflate, Zstandard, and Oodle algorithms. 2. **RSZ Reflection System Deserialization:** - Parse `RSZ` data blocks: magic header, object count, instance table, and element data blocks. - Map 32-bit type hashes to human-readable class definitions and field types (int, float, string, boolean, enum, object references). - Reconstruct cyclic and polymorphic object graphs without native engine binaries. 3. **Scene & Prefab Reconstruction (`.scn`, `.pfb`):** - Traverse `GameObject` hierarchies, parent-child folder structures, and active components (e.g., renderers, colliders, audio sources, script controllers). - Reconstruct prefab component overrides and property variations. 4. **Material & Texture Extraction:** - Parse `.mdf2` material files: extract shader property uniforms, texture slot bindings (albedo, normal, roughness, metallic, ambient occlusion). - Decode `.tex` texture files: convert BC1, BC4, BC5, and BC7 compressed surfaces to PNG/DDS. ### Acceptance Criteria - [ ] Parsing and safe file extraction of `KPKA`-formatted archive containers. - [ ] Deserialization of `RSZ` binary data into structured, typed JSON object graphs. - [ ] Traversal of `.scn` and `.pfb` hierarchical component trees. - [ ] Extraction and decoding of `.tex` textures and `.mdf2` material definitions. - [ ] Export of an end-to-end scene and component relationship diagram. ### Related Issues & Dependencies - Complements #3621 (Unknown binary file-format inference) - Complements #3624 (Native C++ semantic reconstruction: RTTI, vtables) - Complements #3698 (Interactive entertainment runtime and asset reverse engineering tracker)

bsakaguchi referenced this issue

2026-10-17 21:11:16 +00:00

[PR #1422] [MERGED] docs: correct host requirements in the README analysis tables #2874

bsakaguchi referenced this issue

2026-10-31 00:50:58 +00:00

[GH-ISSUE #7656] [Bug] Indonesian and Italian analysis tables omit the Linux x64 host prerequisite #4579