[GH-ISSUE #3648] [Feature] Ren'Py reverse engineering: RPA archives, RPYC compiled scripts, screens/ATL, labels and route reconstruction #1366

New issue

Open

opened 2026-10-16 21:04:43 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:04:43 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3648

Summary

The Ren'Py Visual Novel Engine powers thousands of interactive visual novels, narrative games (Doki Doki Literature Club!, Slay the Princess, Katawa Shoujo), and educational simulations on desktop and mobile. Packaged Ren'Py games bundle their dialogue, branching routes, Python logic, screens, and multimedia assets into compiled and archived formats:

RPA Archives (.rpa): Ren'Py Archive files (RPA-1.0, RPA-2.0, RPA-3.0, RPA-3.2) holding images, audio, and scripts with obfuscated key indexes;
Compiled Scripts (.rpyc, .rpymc): Compiled Ren'Py scripts containing zlib-compressed Python pickle streams encoding abstract syntax tree (AST) nodes;
Script Nodes: Labels, dialogues (say), menus (menu), choices, jumps (jump), calls (call), and conditional branches (if/elif/else);
Screen Language & ATL: User interface screens (screen), displayables, and Animation & Transformation Language (atl) blocks;
Python Integration: Python code blocks (init python:, $ var = ...) manipulating persistent game state.

Currently, REA does not cover Ren'Py game bundles or .rpyc files. While issue #2296 and #2383 discuss Python runtime tracking, Ren'Py requires dedicated static AST de-serialization and route reconstruction.

This feature proposal introduces dedicated Ren'Py Reverse Engineering to REA, enabling RPA archive extraction, .rpyc AST de-pickling without code execution, screen/ATL reconstruction, and dialogue/route graph generation.

Technical Capabilities & Architecture

                                    +-----------------------------------------+
                                    | Input: Ren'Py Game Directory / .rpa /   |
                                    |        compiled .rpyc scripts           |
                                    +--------------------+--------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | RPA Archive Unpacker      |                                   | Safe .rpyc AST Reader     |
           | (RPA 1.0, 2.0, 3.0, 3.2)  |                                   | (Pickle AST / No Exec)    |
           +-------------+-------------+                                   +-------------+-------------+
                         |                                                               |
                         +-------------------------------+-------------------------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | Script AST & Node Decompiler          |
                                     | (Say, Menu, Jump, Call, Python blocks)|
                                     +-------------------+-------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | Dialogue & Branching      |                                   | Screen Language & ATL     |
           | Route Graph (Flowchart)   |                                   | Reconstruction (UI/Anims) |
           +-------------+-------------+                                   +-------------+-------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | REA Evidence: Full Script Text,       |
                                     | Route Decision Graph & Endings        |
                                     +---------------------------------------+

1. RPA Archive Unpacker (RPA 1.0, 2.0, 3.0, 3.2)

Header Parsing: Parse RPA container formats:
- RPA-1.0: raw archive files.
- RPA-2.0 / 3.0: header line RPA-3.0 <index_offset> <key>, extracting index offset and 32-bit XOR key.
- RPA-3.2: obfuscated index structures.
Index Decompression & File Extraction: Decompress zlib-compressed index streams, apply XOR key transformations to file offsets and lengths, and extract original files (.rpyc, .png, .webp, .ogg, .mp3) preserving directory hierarchies.

2. Safe `.rpyc` / `.rpymc` AST De-serialization (Without `pickle.load`)

Format Header: Parse Ren'Py compiled script header (RENPY RPC-2 magic), extracting format version numbers and zlib-compressed pickle data slices.
Inert AST Parser: Statically parse Python pickle opcodes without executing pickle.load (preventing arbitrary code execution from malicious pickles).
Ren'Py AST Node Mapping: Map de-serialized objects to known Ren'Py AST classes:
- renpy.ast.Say: dialogue line, speaker name, text string, transition arguments.
- renpy.ast.Menu: choice menus, choice conditions, jump targets.
- renpy.ast.Label: label identifiers, parameters.
- renpy.ast.Jump / renpy.ast.Call: control flow transfers.
- renpy.ast.If: conditional branches (entries list with condition expressions).
- renpy.ast.Python: embedded Python code snippets.
- renpy.ast.Show / renpy.ast.Scene / renpy.ast.Hide: visual staging commands.

3. Decompilation into Clean Ren'Py Script (`.rpy`)

Script Reconstruction: Invert recovered AST nodes into clean, human-readable .rpy source code:
- Reconstruct speaker character aliases and formatting tags ({b}, {i}, {color}).
- Reconstruct menu choices, indentation blocks, and variable modifications ($ flag = True).

4. Branching Route Graph & Endings Reconstruction

Route Traversal: Build a complete directed graph of the narrative:
- Nodes: Label, Menu choice points, narrative endings (return).
- Edges: jump, call, and menu selection transitions annotated with condition requirements.
Ending & Flag Analysis: Identify all reachable endings (good, bad, true endings) and map required flags/variables to achieve each path.

5. Screen Language & ATL Analysis

Screen Definitions: Reconstruct screen blocks (buttons, imagemaps, text inputs, viewports, frames).
Animation & Transformation Language (ATL): Parse atl blocks defining sprite movements, zooms, rotations, and alpha fades.

Acceptance Criteria

Introduce analyze_renpy_game in REA CLI and MCP contracts, accepting game directories, .rpa archives, and .rpyc files.
Unpack synthetic RPA-2.0 and RPA-3.0 archives with verified file digests.
Safely de-serialize .rpyc test fixtures into Ren'Py AST nodes without executing arbitrary Python pickle code.
Decompile synthetic .rpyc files into valid .rpy script text.
Export a complete narrative route decision graph in structured JSON evidence.

#2296 [Feature] Multi-process Python application evidence
#2383 [Tracker] Audit and subsystem index of active bug reports
#3632 [Feature] GameMaker reverse engineering: data.win, GML bytecode

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3648 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ## Summary The Ren'Py Visual Novel Engine powers thousands of interactive visual novels, narrative games (*Doki Doki Literature Club!*, *Slay the Princess*, *Katawa Shoujo*), and educational simulations on desktop and mobile. Packaged Ren'Py games bundle their dialogue, branching routes, Python logic, screens, and multimedia assets into compiled and archived formats: - **RPA Archives (`.rpa`):** Ren'Py Archive files (RPA-1.0, RPA-2.0, RPA-3.0, RPA-3.2) holding images, audio, and scripts with obfuscated key indexes; - **Compiled Scripts (`.rpyc`, `.rpymc`):** Compiled Ren'Py scripts containing zlib-compressed Python pickle streams encoding abstract syntax tree (AST) nodes; - **Script Nodes:** Labels, dialogues (`say`), menus (`menu`), choices, jumps (`jump`), calls (`call`), and conditional branches (`if`/`elif`/`else`); - **Screen Language & ATL:** User interface screens (`screen`), displayables, and Animation & Transformation Language (`atl`) blocks; - **Python Integration:** Python code blocks (`init python:`, `$ var = ...`) manipulating persistent game state. Currently, REA does not cover Ren'Py game bundles or `.rpyc` files. While issue #2296 and #2383 discuss Python runtime tracking, Ren'Py requires dedicated static AST de-serialization and route reconstruction. This feature proposal introduces dedicated **Ren'Py Reverse Engineering** to REA, enabling RPA archive extraction, `.rpyc` AST de-pickling without code execution, screen/ATL reconstruction, and dialogue/route graph generation. --- ## Technical Capabilities & Architecture ``` +-----------------------------------------+ | Input: Ren'Py Game Directory / .rpa / | | compiled .rpyc scripts | +--------------------+--------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | RPA Archive Unpacker | | Safe .rpyc AST Reader | | (RPA 1.0, 2.0, 3.0, 3.2) | | (Pickle AST / No Exec) | +-------------+-------------+ +-------------+-------------+ | | +-------------------------------+-------------------------------+ | v +---------------------------------------+ | Script AST & Node Decompiler | | (Say, Menu, Jump, Call, Python blocks)| +-------------------+-------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | Dialogue & Branching | | Screen Language & ATL | | Route Graph (Flowchart) | | Reconstruction (UI/Anims) | +-------------+-------------+ +-------------+-------------+ | v +---------------------------------------+ | REA Evidence: Full Script Text, | | Route Decision Graph & Endings | +---------------------------------------+ ``` ### 1. RPA Archive Unpacker (RPA 1.0, 2.0, 3.0, 3.2) - **Header Parsing:** Parse RPA container formats: - RPA-1.0: raw archive files. - RPA-2.0 / 3.0: header line `RPA-3.0 <index_offset> <key>`, extracting index offset and 32-bit XOR key. - RPA-3.2: obfuscated index structures. - **Index Decompression & File Extraction:** Decompress zlib-compressed index streams, apply XOR key transformations to file offsets and lengths, and extract original files (`.rpyc`, `.png`, `.webp`, `.ogg`, `.mp3`) preserving directory hierarchies. ### 2. Safe `.rpyc` / `.rpymc` AST De-serialization (Without `pickle.load`) - **Format Header:** Parse Ren'Py compiled script header (`RENPY RPC-2` magic), extracting format version numbers and zlib-compressed pickle data slices. - **Inert AST Parser:** Statically parse Python pickle opcodes without executing `pickle.load` (preventing arbitrary code execution from malicious pickles). - **Ren'Py AST Node Mapping:** Map de-serialized objects to known Ren'Py AST classes: - `renpy.ast.Say`: dialogue line, speaker name, text string, transition arguments. - `renpy.ast.Menu`: choice menus, choice conditions, jump targets. - `renpy.ast.Label`: label identifiers, parameters. - `renpy.ast.Jump` / `renpy.ast.Call`: control flow transfers. - `renpy.ast.If`: conditional branches (`entries` list with condition expressions). - `renpy.ast.Python`: embedded Python code snippets. - `renpy.ast.Show` / `renpy.ast.Scene` / `renpy.ast.Hide`: visual staging commands. ### 3. Decompilation into Clean Ren'Py Script (`.rpy`) - **Script Reconstruction:** Invert recovered AST nodes into clean, human-readable `.rpy` source code: - Reconstruct speaker character aliases and formatting tags (`{b}`, `{i}`, `{color}`). - Reconstruct menu choices, indentation blocks, and variable modifications (`$ flag = True`). ### 4. Branching Route Graph & Endings Reconstruction - **Route Traversal:** Build a complete directed graph of the narrative: - Nodes: `Label`, `Menu` choice points, narrative endings (`return`). - Edges: `jump`, `call`, and menu selection transitions annotated with condition requirements. - **Ending & Flag Analysis:** Identify all reachable endings (good, bad, true endings) and map required flags/variables to achieve each path. ### 5. Screen Language & ATL Analysis - **Screen Definitions:** Reconstruct `screen` blocks (buttons, imagemaps, text inputs, viewports, frames). - **Animation & Transformation Language (ATL):** Parse `atl` blocks defining sprite movements, zooms, rotations, and alpha fades. --- ## Acceptance Criteria 1. Introduce `analyze_renpy_game` in REA CLI and MCP contracts, accepting game directories, `.rpa` archives, and `.rpyc` files. 2. Unpack synthetic RPA-2.0 and RPA-3.0 archives with verified file digests. 3. Safely de-serialize `.rpyc` test fixtures into Ren'Py AST nodes without executing arbitrary Python pickle code. 4. Decompile synthetic `.rpyc` files into valid `.rpy` script text. 5. Export a complete narrative route decision graph in structured JSON evidence. --- ## Related - #2296 [Feature] Multi-process Python application evidence - #2383 [Tracker] Audit and subsystem index of active bug reports - #3632 [Feature] GameMaker reverse engineering: data.win, GML bytecode

bsakaguchi referenced this issue

2026-10-17 21:01:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:10:47 +00:00

[PR #1366] [MERGED] fix(mcp): preserve JavaScript workflow identity validation #2830