YFT models and YTD texture dictionaries #1411

New issue

Open

opened 2026-10-16 21:05:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:05:13 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3701

Problem Statement & Context

Advanced open-world engines package game worlds, physical simulations, entity models, textures, audio, and compiled mission scripts into proprietary hierarchical archive containers (.rpf) paired with custom resource-streaming headers (RSC7 / RSC8).

Analyzing these containers and their packed resources presents major reverse engineering challenges:

Container Headers & Virtual Filesystems (.rpf v7 / v8): Multi-level Table of Contents (TOC) entries defining directories, physical files, and resource files. Supports encryption (AES-256 / custom table permutations) and Deflate compression.
Resource Streaming Headers (RSC7 / RSC8): Instead of raw file data, streaming assets are stored as pre-mapped virtual memory pages divided into system pages (CPU RAM) and graphics pages (VRAM GPU memory), preceded by a 16-byte RSC header defining version and page split flags.
Compiled Script Bytecode (.ysc / .ysc.full): Custom stack-based virtual machine executing opcodes for arithmetic, global/local variable access, struct manipulation, string lookups, and native function calls via 64-bit hashes.
Texture Dictionaries (.ytd): Packages containing compressed texture surfaces (DXT1, DXT3, DXT5, BC7) and mipmap chains.
Drawable Models & Fragments (.ydr, .yft): Mesh geometry, bone skeletons, shader parameter definitions, and physical breakable fragment hierarchies.

REA lacks a native parser for .rpf container trees and the memory-mapped RSC streaming deserialization pipeline.

Proposed Architecture

flowchart TD
    A[".rpf Archive File (v7 / v8)"] --> B["RPF Archive & Resource Engine"]
    B --> C["TOC & Directory Tree Deserializer"]
    B --> D["RSC Memory Page Reconstructor"]
    B --> E["Domain Asset Analyzers"]
    
    C --> C1["Header Magic Verification (RPF7 / RPF8)"]
    C --> C2["TOC Entries (Directory / Binary File / Resource File)"]
    C --> C3["Decompression (Deflate / Zlib) & Extraction"]
    
    D --> D1["RSC Header Parsing (Version, System/Graphics Page Flags)"]
    D --> D2["Virtual Memory Pointer Fixup & Address Translation"]
    
    E --> E1[".ysc Script Bytecode Disassembler"]
    E --> E2[".ytd Texture Dictionary Extractor (DDS / PNG)"]
    E --> E3[".ydr / .yft Drawable & Fragment Geometry Inspector"]
    E --> E4[".ymap / .ytyp Spatial Entity Placers & Archetypes"]
    
    C & D & E --> F["Unified Archive Blueprint, Decompiled Scripts & Extracted Assets"]

Technical Capabilities & Modules

RPF Container Parsing & File Tree Extraction:
- Parse RPF7 (v7) and RPF8 (v8) archive header magic, entry counts, and TOC metadata.
- Reconstruct the nested directory tree and distinguish standard binary files from RSC resource files.
- Decompress Deflate-compressed streams safely to disk.
RSC Memory Page Fixup & Deserialization:
- Parse 16-byte RSC7 / RSC8 resource headers.
- Read virtual memory page allocation tables (system memory chunks vs graphics memory chunks).
- Resolve base pointers and fix up relative memory pointers to reconstitute typed C++ struct instances.
Compiled Script Bytecode Disassembly (.ysc):
- Disassemble custom script bytecode instructions (arithmetic, stack manipulation, jumps, calls).
- Map script parameters, static variables, and function boundaries.
- Correlate 64-bit native invocation hashes with engine API function signatures.
Texture Dictionary Extraction (.ytd):
- Deserialize texture dictionary structures from graphics memory pages.
- Extract mipmapped texture surfaces and export to standard .dds or .png formats.
Spatial Placement & Entity Archetype Analysis (.ymap, .ytyp):
- Parse placement definitions: entity positions, rotations, scale, LOD distances, and model archetype references.
- Reconstruct spatial scene graphs detailing entity layouts.

Acceptance Criteria

Automated parsing and unpacking of RPF7 and RPF8 archive containers.
Safe decompression and extraction of regular binary files and RSC resource files.
Implementation of RSC system/graphics virtual memory pointer fixups.
Disassembly of .ysc script bytecode with native hash resolution.
Extraction of textures from .ytd dictionaries into standard image formats.

Complements #3621 (Unknown binary file-format inference)
Complements #3624 (Native C++ semantic reconstruction: RTTI, vtables)
Complements #3699 (Modular multiplayer resource package reverse engineering)

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3701 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ### Problem Statement & Context Advanced open-world engines package game worlds, physical simulations, entity models, textures, audio, and compiled mission scripts into proprietary hierarchical archive containers (`.rpf`) paired with custom resource-streaming headers (`RSC7` / `RSC8`). Analyzing these containers and their packed resources presents major reverse engineering challenges: 1. **Container Headers & Virtual Filesystems (`.rpf` v7 / v8)**: Multi-level Table of Contents (TOC) entries defining directories, physical files, and resource files. Supports encryption (AES-256 / custom table permutations) and Deflate compression. 2. **Resource Streaming Headers (`RSC7` / `RSC8`)**: Instead of raw file data, streaming assets are stored as pre-mapped virtual memory pages divided into system pages (CPU RAM) and graphics pages (VRAM GPU memory), preceded by a 16-byte `RSC` header defining version and page split flags. 3. **Compiled Script Bytecode (`.ysc` / `.ysc.full`)**: Custom stack-based virtual machine executing opcodes for arithmetic, global/local variable access, struct manipulation, string lookups, and native function calls via 64-bit hashes. 4. **Texture Dictionaries (`.ytd`)**: Packages containing compressed texture surfaces (DXT1, DXT3, DXT5, BC7) and mipmap chains. 5. **Drawable Models & Fragments (`.ydr`, `.yft`)**: Mesh geometry, bone skeletons, shader parameter definitions, and physical breakable fragment hierarchies. REA lacks a native parser for `.rpf` container trees and the memory-mapped `RSC` streaming deserialization pipeline. ### Proposed Architecture ```mermaid flowchart TD A[".rpf Archive File (v7 / v8)"] --> B["RPF Archive & Resource Engine"] B --> C["TOC & Directory Tree Deserializer"] B --> D["RSC Memory Page Reconstructor"] B --> E["Domain Asset Analyzers"] C --> C1["Header Magic Verification (RPF7 / RPF8)"] C --> C2["TOC Entries (Directory / Binary File / Resource File)"] C --> C3["Decompression (Deflate / Zlib) & Extraction"] D --> D1["RSC Header Parsing (Version, System/Graphics Page Flags)"] D --> D2["Virtual Memory Pointer Fixup & Address Translation"] E --> E1[".ysc Script Bytecode Disassembler"] E --> E2[".ytd Texture Dictionary Extractor (DDS / PNG)"] E --> E3[".ydr / .yft Drawable & Fragment Geometry Inspector"] E --> E4[".ymap / .ytyp Spatial Entity Placers & Archetypes"] C & D & E --> F["Unified Archive Blueprint, Decompiled Scripts & Extracted Assets"] ``` ### Technical Capabilities & Modules 1. **RPF Container Parsing & File Tree Extraction:** - Parse RPF7 (v7) and RPF8 (v8) archive header magic, entry counts, and TOC metadata. - Reconstruct the nested directory tree and distinguish standard binary files from `RSC` resource files. - Decompress Deflate-compressed streams safely to disk. 2. **RSC Memory Page Fixup & Deserialization:** - Parse 16-byte `RSC7` / `RSC8` resource headers. - Read virtual memory page allocation tables (system memory chunks vs graphics memory chunks). - Resolve base pointers and fix up relative memory pointers to reconstitute typed C++ struct instances. 3. **Compiled Script Bytecode Disassembly (`.ysc`):** - Disassemble custom script bytecode instructions (arithmetic, stack manipulation, jumps, calls). - Map script parameters, static variables, and function boundaries. - Correlate 64-bit native invocation hashes with engine API function signatures. 4. **Texture Dictionary Extraction (`.ytd`):** - Deserialize texture dictionary structures from graphics memory pages. - Extract mipmapped texture surfaces and export to standard `.dds` or `.png` formats. 5. **Spatial Placement & Entity Archetype Analysis (`.ymap`, `.ytyp`):** - Parse placement definitions: entity positions, rotations, scale, LOD distances, and model archetype references. - Reconstruct spatial scene graphs detailing entity layouts. ### Acceptance Criteria - [ ] Automated parsing and unpacking of RPF7 and RPF8 archive containers. - [ ] Safe decompression and extraction of regular binary files and RSC resource files. - [ ] Implementation of RSC system/graphics virtual memory pointer fixups. - [ ] Disassembly of `.ysc` script bytecode with native hash resolution. - [ ] Extraction of textures from `.ytd` dictionaries into standard image formats. ### Related Issues & Dependencies - Complements #3621 (Unknown binary file-format inference) - Complements #3624 (Native C++ semantic reconstruction: RTTI, vtables) - Complements #3699 (Modular multiplayer resource package reverse engineering)

bsakaguchi referenced this issue

2026-10-17 21:11:11 +00:00

[PR #1411] [CLOSED] fix(browser): resolve authorized scope when Chrome masks frame URLs #2865

bsakaguchi referenced this issue

2026-10-17 21:11:22 +00:00

[PR #1432] [CLOSED] fix(browser): launch verification Chrome without the desktop keyring #2881