morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3625] [Feature] Unreal Engine reverse engineering: Pak/IoStore, UAsset reflection, and Blueprint bytecode reconstruction #1352

New issue

Open

opened 2026-10-16 21:04:35 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:04:35 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3625

Summary

Unreal Engine (UE4 and UE5) is one of the premier game engines and real-time 3D platforms, powering AAA games, simulations, automotive interfaces, and virtual production. While Unity reverse engineering is addressed in #3607, Unreal Engine employs an entirely distinct binary architecture, asset serialization model, and virtual machine.

In packaged Unreal Engine games, assets and logic are compiled into tightly optimized containers and binary packages:

Container Archives: Legacy Unreal Pak (.pak) files and modern UE5 Zen / IoStore containers (.utoc Table of Contents + .ucas Container CAS files).
Serialized Asset Packages: Split asset files (.uasset package headers, .uexp export payloads, .ubulk uncompressed bulk data buffers).
Object Model & Reflection: UObject, UClass, FProperty, UFunction, UScriptStruct, and UEnum reflection definitions.
Asset Registry & Data Tables: AssetRegistry.bin global metadata and UDataTable structured game data.
Blueprint / Kismet Bytecode: Blueprint visual scripting graphs compiled into stack-based Kismet virtual machine bytecode.

This feature proposal introduces dedicated Unreal Engine Reverse Engineering to REA, enabling container unpacking (Pak and IoStore), UAsset reflection recovery, DataTable extraction, and Blueprint bytecode disassembly/decompilation.

Technical Capabilities & Architecture

                                    +-----------------------------------------+
                                    | Input: Unreal Engine Game Directory /   |
                                    |        .pak, .utoc, .ucas, .uasset      |
                                    +--------------------+--------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | Pak & IoStore Container   |                                   | AssetRegistry & Name Map  |
           | Unpacker (.pak, .utoc/.ucas)                                  | Parsing (AssetRegistry.bin)|
           +-------------+-------------+                                   +-------------+-------------+
                         |                                                               |
                         +-------------------------------+-------------------------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | UAsset Package Parser                 |
                                     | (.uasset, .uexp, .ubulk, Name/Imports)|
                                     +-------------------+-------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | Blueprint / Kismet        |                                   | DataTable & Struct        |
           | Bytecode Decompiler       |                                   | Extraction (JSON Data)    |
           +-------------+-------------+                                   +-------------+-------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | REA Evidence & Reconstructed Artifacts|
                                     | (Pseudocode, Schemas, Asset Trees)    |
                                     +---------------------------------------+

1. Container Unpacking: Pak & IoStore (Zen)

Unreal Pak Files (.pak):
- Support Pak format versions (v1 through v11+), parsing index footers, mount points, and encrypted/compressed chunk records.
- Decompress standard compression algorithms: Oodle (Kraken, Leviathan, Mermaid, Selkie), Zlib, and LZ4.
- Support caller-supplied AES-256 decryption keys for encrypted Pak headers and chunk payloads.
IoStore / Zen Containers (.utoc + .ucas):
- Parse modern UE5 IoStore containers: FIoStoreTocHeader, FIoChunkId, container flags, and chunk compression blocks.
- Reconstruct directory paths from FDirectoryIndexStore.

2. UAsset Package & Reflection Serialization

Split Asset Reader (.uasset + .uexp + .ubulk):
- Parse FPackageFileSummary: package flags, total header size, custom version GUIDs, and engine version tags.
- Extract Name Map: decode serialized FNameEntry / FNameEntrySerialized string tables.
- Extract Import Table (FObjectImport): resolve dependencies to external packages, engine classes, and native modules.
- Extract Export Table (FObjectExport): map internal object definitions, class indices, super indices, and memory offsets in .uexp.
Reflection Metadata Extraction:
- Reconstruct UClass hierarchies, struct layouts (UScriptStruct), and enum definitions (UEnum).
- Decode property types: numeric primitives, FStrProperty, FTextProperty, FNameProperty, FArrayProperty, FMapProperty, FStructProperty, and FObjectProperty pointers.

3. DataTable & String Asset Extraction

DataTable (UDataTable) Rows: Deserialize binary row structs into structured, human-readable JSON tables (e.g. weapon statistics, dialogue trees, game balance curves, drop tables).
Localization & String Tables: Extract localized text databases (.locres / .locmeta) and UStringTable assets.
Global Asset Registry (AssetRegistry.bin): Parse the global asset registry to extract package metadata, asset tags, primary asset bundles, and cross-package dependency graphs without inspecting individual asset payloads.

4. Blueprint / Kismet Bytecode Disassembly & Decompilation

Script Bytecode Extraction: Extract bytecode instruction streams stored in UFunction and UBlueprintGeneratedClass records.
Kismet Opcode Disassembly:
- Disassemble Kismet VM opcodes: variable access (EX_LocalVariable, EX_InstanceVariable), assignments (EX_Let), mathematical operations (EX_CallMath), jumps/branches (EX_Jump, EX_JumpIfNot), dynamic casts (EX_DynamicCast), and function invocations (EX_VirtualFunction, EX_FinalFunction, EX_LocalVirtualFunction).
High-Level Decompilation: Reconstruct control flow graphs into structured, high-level pseudocode or C++-like representations of the original Blueprint node graphs.

5. Media & Texture Bulk Carving

Texture Payloads: Extract raw texture compression streams (UTexture2D in DXT1, DXT5, BC7, ASTC, RGBA) from .ubulk buffers.
Audio Payloads: Extract embedded audio data (USoundWave / Opus, ADPCM, Bink Audio) with sample rate and channel layout metadata.

Acceptance Criteria

Introduce analyze_unreal_package in REA CLI and MCP contracts, accepting .pak, .utoc, and .uasset inputs.
Safely unpack synthetic .pak and IoStore .utoc/.ucas archives without memory leaks or uncapped allocations.
Parse .uasset + .uexp files to extract Name Maps, Import/Export tables, and object schemas into structured JSON.
Deserialize synthetic UDataTable binary assets into valid JSON row arrays.
Disassemble and decompile Kismet bytecode from a synthetic UFunction into human-readable pseudocode.
Enforce safe limits on Oodle/Zlib decompression and handle missing AES keys with typed decryption_key_required diagnostics.

#3607 [Feature] Comprehensive Unity extraction and reverse engineering
#743 [Feature] Recover .NET NativeAOT metadata in Ghidra workflows
#3620 [Feature] Shader and GPU Bytecode Reverse Engineering
#3624 [Feature] Native C++ semantic reconstruction

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3625 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ## Summary Unreal Engine (UE4 and UE5) is one of the premier game engines and real-time 3D platforms, powering AAA games, simulations, automotive interfaces, and virtual production. While Unity reverse engineering is addressed in #3607, Unreal Engine employs an entirely distinct binary architecture, asset serialization model, and virtual machine. In packaged Unreal Engine games, assets and logic are compiled into tightly optimized containers and binary packages: - **Container Archives:** Legacy Unreal Pak (`.pak`) files and modern UE5 Zen / IoStore containers (`.utoc` Table of Contents + `.ucas` Container CAS files). - **Serialized Asset Packages:** Split asset files (`.uasset` package headers, `.uexp` export payloads, `.ubulk` uncompressed bulk data buffers). - **Object Model & Reflection:** `UObject`, `UClass`, `FProperty`, `UFunction`, `UScriptStruct`, and `UEnum` reflection definitions. - **Asset Registry & Data Tables:** `AssetRegistry.bin` global metadata and `UDataTable` structured game data. - **Blueprint / Kismet Bytecode:** Blueprint visual scripting graphs compiled into stack-based Kismet virtual machine bytecode. This feature proposal introduces dedicated **Unreal Engine Reverse Engineering** to REA, enabling container unpacking (Pak and IoStore), UAsset reflection recovery, DataTable extraction, and Blueprint bytecode disassembly/decompilation. --- ## Technical Capabilities & Architecture ``` +-----------------------------------------+ | Input: Unreal Engine Game Directory / | | .pak, .utoc, .ucas, .uasset | +--------------------+--------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | Pak & IoStore Container | | AssetRegistry & Name Map | | Unpacker (.pak, .utoc/.ucas) | Parsing (AssetRegistry.bin)| +-------------+-------------+ +-------------+-------------+ | | +-------------------------------+-------------------------------+ | v +---------------------------------------+ | UAsset Package Parser | | (.uasset, .uexp, .ubulk, Name/Imports)| +-------------------+-------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | Blueprint / Kismet | | DataTable & Struct | | Bytecode Decompiler | | Extraction (JSON Data) | +-------------+-------------+ +-------------+-------------+ | v +---------------------------------------+ | REA Evidence & Reconstructed Artifacts| | (Pseudocode, Schemas, Asset Trees) | +---------------------------------------+ ``` ### 1. Container Unpacking: Pak & IoStore (Zen) - **Unreal Pak Files (`.pak`):** - Support Pak format versions (v1 through v11+), parsing index footers, mount points, and encrypted/compressed chunk records. - Decompress standard compression algorithms: Oodle (Kraken, Leviathan, Mermaid, Selkie), Zlib, and LZ4. - Support caller-supplied AES-256 decryption keys for encrypted Pak headers and chunk payloads. - **IoStore / Zen Containers (`.utoc` + `.ucas`):** - Parse modern UE5 IoStore containers: `FIoStoreTocHeader`, `FIoChunkId`, container flags, and chunk compression blocks. - Reconstruct directory paths from `FDirectoryIndexStore`. ### 2. UAsset Package & Reflection Serialization - **Split Asset Reader (`.uasset` + `.uexp` + `.ubulk`):** - Parse `FPackageFileSummary`: package flags, total header size, custom version GUIDs, and engine version tags. - Extract **Name Map:** decode serialized `FNameEntry` / `FNameEntrySerialized` string tables. - Extract **Import Table (`FObjectImport`):** resolve dependencies to external packages, engine classes, and native modules. - Extract **Export Table (`FObjectExport`):** map internal object definitions, class indices, super indices, and memory offsets in `.uexp`. - **Reflection Metadata Extraction:** - Reconstruct `UClass` hierarchies, struct layouts (`UScriptStruct`), and enum definitions (`UEnum`). - Decode property types: numeric primitives, `FStrProperty`, `FTextProperty`, `FNameProperty`, `FArrayProperty`, `FMapProperty`, `FStructProperty`, and `FObjectProperty` pointers. ### 3. DataTable & String Asset Extraction - **DataTable (`UDataTable`) Rows:** Deserialize binary row structs into structured, human-readable JSON tables (e.g. weapon statistics, dialogue trees, game balance curves, drop tables). - **Localization & String Tables:** Extract localized text databases (`.locres` / `.locmeta`) and `UStringTable` assets. - **Global Asset Registry (`AssetRegistry.bin`):** Parse the global asset registry to extract package metadata, asset tags, primary asset bundles, and cross-package dependency graphs without inspecting individual asset payloads. ### 4. Blueprint / Kismet Bytecode Disassembly & Decompilation - **Script Bytecode Extraction:** Extract bytecode instruction streams stored in `UFunction` and `UBlueprintGeneratedClass` records. - **Kismet Opcode Disassembly:** - Disassemble Kismet VM opcodes: variable access (`EX_LocalVariable`, `EX_InstanceVariable`), assignments (`EX_Let`), mathematical operations (`EX_CallMath`), jumps/branches (`EX_Jump`, `EX_JumpIfNot`), dynamic casts (`EX_DynamicCast`), and function invocations (`EX_VirtualFunction`, `EX_FinalFunction`, `EX_LocalVirtualFunction`). - **High-Level Decompilation:** Reconstruct control flow graphs into structured, high-level pseudocode or C++-like representations of the original Blueprint node graphs. ### 5. Media & Texture Bulk Carving - **Texture Payloads:** Extract raw texture compression streams (`UTexture2D` in DXT1, DXT5, BC7, ASTC, RGBA) from `.ubulk` buffers. - **Audio Payloads:** Extract embedded audio data (`USoundWave` / Opus, ADPCM, Bink Audio) with sample rate and channel layout metadata. --- ## Acceptance Criteria 1. Introduce `analyze_unreal_package` in REA CLI and MCP contracts, accepting `.pak`, `.utoc`, and `.uasset` inputs. 2. Safely unpack synthetic `.pak` and IoStore `.utoc`/`.ucas` archives without memory leaks or uncapped allocations. 3. Parse `.uasset` + `.uexp` files to extract Name Maps, Import/Export tables, and object schemas into structured JSON. 4. Deserialize synthetic `UDataTable` binary assets into valid JSON row arrays. 5. Disassemble and decompile Kismet bytecode from a synthetic `UFunction` into human-readable pseudocode. 6. Enforce safe limits on Oodle/Zlib decompression and handle missing AES keys with typed `decryption_key_required` diagnostics. --- ## Related - #3607 [Feature] Comprehensive Unity extraction and reverse engineering - #743 [Feature] Recover .NET NativeAOT metadata in Ghidra workflows - #3620 [Feature] Shader and GPU Bytecode Reverse Engineering - #3624 [Feature] Native C++ semantic reconstruction

bsakaguchi referenced this issue

2026-10-17 21:10:38 +00:00

[PR #1352] [CLOSED] fix(setup): link managed skill into Claude Code skills directory #2819

bsakaguchi commented

2026-10-31 00:30:43 +00:00

Author

Owner

@8rulerstar commented on GitHub (Oct 20, 2026):

For reference, #4362 is an open PR for read-only .pak inventory (pak versions 1 to 12: footer, index, entries, compression and encryption flags), compared against repak on a UE 5.8.3 pak.

@8rulerstar commented on GitHub (Oct 20, 2026): For reference, #4362 is an open PR for read-only `.pak` inventory (pak versions 1 to 12: footer, index, entries, compression and encryption flags), compared against repak on a UE 5.8.3 pak.

bsakaguchi commented

2026-10-31 00:30:43 +00:00

Author

Owner

@DarkNightForge commented on GitHub (Oct 21, 2026):

I'd like to take the Blueprint/Kismet bytecode half of this, if the scope fits.

#4362 covers .pak inventory, and its author has said extraction, decryption, property
deserialisation and Blueprint bytecode are outside what they plan to do. That leaves the bytecode
side unclaimed. I work in UE 5.8 regularly, so I know what UFunction, UBlueprintGeneratedClass
and the cook output actually are, and — more importantly for verification — I can cook fixtures from
a project I own rather than working from someone else's shipped game.

Analyst question. "What does this Blueprint actually do?" Today REA detects Unreal (the
Content/Paks convention and the IoStore TOC magic in src/domain/nativeConvention.ts) and stops
there. A packaged UE title's gameplay logic is compiled Kismet bytecode living inside UFunction
exports in .uexp. No existing REA operation can read it, and the JavaScript, native and managed
lanes don't apply to it.

Proposed scope — one operation, read-only.

Input: a .uasset / .uexp pair plus an export name.
Output: a decoded instruction list — EExprToken opcode, operands, byte offsets — as Evidence.
Shape: src/domain/unreal/kismetBytecode.ts as a pure decoder, plus its tool contract.

Non-goals, explicitly. No decompilation to pseudocode — if that is ever wanted it should be a
separate decision. No extraction, decryption or asset recovery. No DRM, anti-cheat or licence-check
surface. Nothing that executes, injects into, mutates or communicates with a target.

Effects. None beyond reading two caller-supplied files: no execution, no injection, no mutation,
no network, no persistence. This is static format decoding, which I read as falling under the
"disassembly, decompilation" line in the supported scope.

Verification. Fixtures cooked from a UE 5.8 project I own, whose Blueprint source I wrote, so the
expected opcodes are known rather than inferred. Goldens generated from those; no third-party or
shipped-game content in the repo. The opcode table comes from EExprToken in the engine's
Script.h.

Sequencing. This needs the .uasset export-map reader from #4362 to locate the bytecode, so I'd
start after that lands. In the meantime, I'd keep any prototype local, to inform the design rather
than to pre-empt the decision.

Could I get a scope decision on that narrow version? Happy to narrow it further if some part of it
doesn't fit.

@DarkNightForge commented on GitHub (Oct 21, 2026): I'd like to take the Blueprint/Kismet bytecode half of this, if the scope fits. #4362 covers `.pak` inventory, and its author has said extraction, decryption, property deserialisation and Blueprint bytecode are outside what they plan to do. That leaves the bytecode side unclaimed. I work in UE 5.8 regularly, so I know what `UFunction`, `UBlueprintGeneratedClass` and the cook output actually are, and — more importantly for verification — I can cook fixtures from a project I own rather than working from someone else's shipped game. **Analyst question.** "What does this Blueprint actually do?" Today REA *detects* Unreal (the `Content/Paks` convention and the IoStore TOC magic in `src/domain/nativeConvention.ts`) and stops there. A packaged UE title's gameplay logic is compiled Kismet bytecode living inside `UFunction` exports in `.uexp`. No existing REA operation can read it, and the JavaScript, native and managed lanes don't apply to it. **Proposed scope — one operation, read-only.** - Input: a `.uasset` / `.uexp` pair plus an export name. - Output: a decoded instruction list — `EExprToken` opcode, operands, byte offsets — as Evidence. - Shape: `src/domain/unreal/kismetBytecode.ts` as a pure decoder, plus its tool contract. **Non-goals, explicitly.** No decompilation to pseudocode — if that is ever wanted it should be a separate decision. No extraction, decryption or asset recovery. No DRM, anti-cheat or licence-check surface. Nothing that executes, injects into, mutates or communicates with a target. **Effects.** None beyond reading two caller-supplied files: no execution, no injection, no mutation, no network, no persistence. This is static format decoding, which I read as falling under the "disassembly, decompilation" line in the supported scope. **Verification.** Fixtures cooked from a UE 5.8 project I own, whose Blueprint source I wrote, so the expected opcodes are known rather than inferred. Goldens generated from those; no third-party or shipped-game content in the repo. The opcode table comes from `EExprToken` in the engine's `Script.h`. **Sequencing.** This needs the `.uasset` export-map reader from #4362 to locate the bytecode, so I'd start after that lands. In the meantime, I'd keep any prototype local, to inform the design rather than to pre-empt the decision. Could I get a scope decision on that narrow version? Happy to narrow it further if some part of it doesn't fit.

bsakaguchi referenced this issue

2026-10-31 00:55:07 +00:00

[PR #1744] [CLOSED] fix(setup): verify independently selected skill destinations #4794

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1352

No description provided.

Rows
Columns