[GH-ISSUE #3683] [Feature] Visual-novel engine reverse engineering: archives, scripts, dialogue trees, flags and route reconstruction #1393

New issue

Open

opened 2026-10-16 21:05:03 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:05:03 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3683

Problem Statement & Context

Visual novels (VNs) represent a prominent class of interactive narrative software, extensively analyzed in game preservation, translation romhacking, and software reverse engineering. While Ren'Py is widely used in Western indie titles (#3648), the broader visual novel ecosystem—particularly Japanese commercial titles and cross-platform ports—is built on specialized, proprietary VN engines with bespoke virtual machines and custom archive formats:

KiriKiri / KAG (吉里吉里): .xp3 archives, KAG script (.ks), and TJS2 compiled bytecode (.tjs).
NScripter / ONScripter: nscript.dat, 0.txt, arc.nsa, default.txt, and byte-shifted bytecode.
CatSystem2: .int archives (blowfish/key encrypted), scene.pack, and custom CST script bytecode.
BGI / Ethornell: .arc containers, Sys3/Sys4 bytecode, and script string encryption.
Majiro: .mjo bytecode scripts and .rc8 archives.
SiglusEngine / RealLive (Key/VisualArts): .g00 graphics, scene.pck, and Key VM bytecode.
Artemis / Yuris / EntisGLS / LiveMaker / Wolf RPG Editor: Proprietary event trees and encrypted asset packs.

Reverse engineering these applications currently requires dozens of disparate community tools (e.g. GARbro, xp3dump, rlvm, kirikiri2 decompiler). REA lacks an extensible, unified framework to unpack proprietary VN archives, decompile script VM bytecodes into structured dialogue trees, resolve branching logic and flag variables, and reconstruct interactive route flowcharts.

Proposed Architecture

flowchart TD
    A["Visual Novel Executable & Game Directory"] --> B["VN Engine Identifier & Unpacker"]
    B --> C["Archive Decryption & Asset Extraction (.xp3, .int, .arc, .nsa)"]
    B --> D["Script Bytecode Decompiler & Virtual Machine Disassembler"]
    
    C --> C1["Images & CG Galleries (.tlg, .hg3, .g00)"]
    C --> C2["Voice Acting & Audio Stems (.ogg, .wav, .opus)"]
    
    D --> D1["Opcode Disassembly (TJS2, CST, MJO, RealLive)"]
    D --> D2["Dialogue & Text String Extraction with Character IDs"]
    D --> D3["Choice & Branch Analysis (Flags, Switches, Affection Points)"]
    
    D1 & D2 & D3 --> E["Interactive Narrative Route Flowchart & Script Reconstruction"]

Technical Capabilities & Modules

Engine Identification & Archive Decryption:
- Signature detection for major VN engines: KiriKiri (Zlib/XP3), CatSystem2 (Blowfish key schedules), BGI/Ethornell, NScripter, Majiro, RealLive/SiglusEngine, Artemis, Yuris, and Buriko General Interpreter.
- Decrypt and unpack proprietary archive formats (.xp3, .int, .arc, .pck, .dat, .nsa).
- Extract and decode specialized image formats: TLG5/TLG6 (KiriKiri), HG-2/HG-3 (CatSystem2), G00/PD (RealLive), and custom mask/alpha formats.
Script Bytecode Disassembly & Decompilation:
- Disassemble custom VM bytecodes into high-level intermediate representation (IR):
  - KiriKiri TJS2 bytecode disassembler and KAG command parser.
  - CatSystem2 CST bytecode parser.
  - Majiro .mjo script decompiler.
  - SiglusEngine / RealLive scenario disassembler.
- De-obfuscate encrypted text strings, shift-JIS / UTF-8 text encodings, and string XOR masks.
Dialogue & Audio Cue Correlation:
- Map lines of dialogue to character speaker IDs, display positions, and expression sprite variants.
- Correlate voice clip filenames (.ogg, .wav, .opus) directly with dialogue line offsets.
- Extract background music (BGM) cues, sound effect (SE) triggers, and screen transitions (fade, wipe, shake).
Narrative Graph & Route Flowchart Reconstruction:
- Extract player choice prompts (select, menu, choice opcodes).
- Trace control flow driven by flag variables, conditions (e.g., affection counters, route completion switches), and jump/call targets.
- Generate Mermaid or Graphviz narrative flowcharts mapping common routes, character routes, branch points, and multiple endings (Good End, True End, Bad End).
Game Assets & CG Gallery Recovery:
- Identify differential sprite layers (base body, clothing variants, facial expressions) and reconstruct composition anchors.
- Map event CG gallery definitions and unlock condition flags.

Acceptance Criteria

Support automated detection and archive extraction for top visual novel engines (KiriKiri XP3, CatSystem2 INT, BGI ARC, NScripter NSA).
Bytecode disassembler recovering clean text dialogue, character speakers, and voice clip associations.
Control-flow analyzer reconstructing player choices, branching flags, and jump destinations.
Automated generation of interactive narrative route graphs depicting branching logic and endings.
Integration with REA asset extraction pipeline for image decoding and audio cue synchronization.

Complements #2383 (Romhacking / translation / localization reverse engineering)
Complements #3648 (Ren'Py reverse engineering: RPA archives, RPYC compiled scripts)
Complements #3647 (RPG Maker reverse engineering)
Complements #3632 (GameMaker reverse engineering)

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3683 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ### Problem Statement & Context Visual novels (VNs) represent a prominent class of interactive narrative software, extensively analyzed in game preservation, translation romhacking, and software reverse engineering. While Ren'Py is widely used in Western indie titles (#3648), the broader visual novel ecosystem—particularly Japanese commercial titles and cross-platform ports—is built on specialized, proprietary VN engines with bespoke virtual machines and custom archive formats: - **KiriKiri / KAG (吉里吉里)**: `.xp3` archives, KAG script (`.ks`), and TJS2 compiled bytecode (`.tjs`). - **NScripter / ONScripter**: `nscript.dat`, `0.txt`, `arc.nsa`, `default.txt`, and byte-shifted bytecode. - **CatSystem2**: `.int` archives (blowfish/key encrypted), `scene.pack`, and custom CST script bytecode. - **BGI / Ethornell**: `.arc` containers, Sys3/Sys4 bytecode, and script string encryption. - **Majiro**: `.mjo` bytecode scripts and `.rc8` archives. - **SiglusEngine / RealLive (Key/VisualArts)**: `.g00` graphics, `scene.pck`, and Key VM bytecode. - **Artemis / Yuris / EntisGLS / LiveMaker / Wolf RPG Editor**: Proprietary event trees and encrypted asset packs. Reverse engineering these applications currently requires dozens of disparate community tools (e.g. GARbro, xp3dump, rlvm, kirikiri2 decompiler). REA lacks an extensible, unified framework to unpack proprietary VN archives, decompile script VM bytecodes into structured dialogue trees, resolve branching logic and flag variables, and reconstruct interactive route flowcharts. ### Proposed Architecture ```mermaid flowchart TD A["Visual Novel Executable & Game Directory"] --> B["VN Engine Identifier & Unpacker"] B --> C["Archive Decryption & Asset Extraction (.xp3, .int, .arc, .nsa)"] B --> D["Script Bytecode Decompiler & Virtual Machine Disassembler"] C --> C1["Images & CG Galleries (.tlg, .hg3, .g00)"] C --> C2["Voice Acting & Audio Stems (.ogg, .wav, .opus)"] D --> D1["Opcode Disassembly (TJS2, CST, MJO, RealLive)"] D --> D2["Dialogue & Text String Extraction with Character IDs"] D --> D3["Choice & Branch Analysis (Flags, Switches, Affection Points)"] D1 & D2 & D3 --> E["Interactive Narrative Route Flowchart & Script Reconstruction"] ``` ### Technical Capabilities & Modules 1. **Engine Identification & Archive Decryption:** - Signature detection for major VN engines: KiriKiri (Zlib/XP3), CatSystem2 (Blowfish key schedules), BGI/Ethornell, NScripter, Majiro, RealLive/SiglusEngine, Artemis, Yuris, and Buriko General Interpreter. - Decrypt and unpack proprietary archive formats (`.xp3`, `.int`, `.arc`, `.pck`, `.dat`, `.nsa`). - Extract and decode specialized image formats: TLG5/TLG6 (KiriKiri), HG-2/HG-3 (CatSystem2), G00/PD (RealLive), and custom mask/alpha formats. 2. **Script Bytecode Disassembly & Decompilation:** - Disassemble custom VM bytecodes into high-level intermediate representation (IR): - KiriKiri TJS2 bytecode disassembler and KAG command parser. - CatSystem2 CST bytecode parser. - Majiro `.mjo` script decompiler. - SiglusEngine / RealLive scenario disassembler. - De-obfuscate encrypted text strings, shift-JIS / UTF-8 text encodings, and string XOR masks. 3. **Dialogue & Audio Cue Correlation:** - Map lines of dialogue to character speaker IDs, display positions, and expression sprite variants. - Correlate voice clip filenames (`.ogg`, `.wav`, `.opus`) directly with dialogue line offsets. - Extract background music (BGM) cues, sound effect (SE) triggers, and screen transitions (fade, wipe, shake). 4. **Narrative Graph & Route Flowchart Reconstruction:** - Extract player choice prompts (`select`, `menu`, `choice` opcodes). - Trace control flow driven by flag variables, conditions (e.g., affection counters, route completion switches), and jump/call targets. - Generate Mermaid or Graphviz narrative flowcharts mapping common routes, character routes, branch points, and multiple endings (Good End, True End, Bad End). 5. **Game Assets & CG Gallery Recovery:** - Identify differential sprite layers (base body, clothing variants, facial expressions) and reconstruct composition anchors. - Map event CG gallery definitions and unlock condition flags. ### Acceptance Criteria - [ ] Support automated detection and archive extraction for top visual novel engines (KiriKiri XP3, CatSystem2 INT, BGI ARC, NScripter NSA). - [ ] Bytecode disassembler recovering clean text dialogue, character speakers, and voice clip associations. - [ ] Control-flow analyzer reconstructing player choices, branching flags, and jump destinations. - [ ] Automated generation of interactive narrative route graphs depicting branching logic and endings. - [ ] Integration with REA asset extraction pipeline for image decoding and audio cue synchronization. ### Related Issues & Dependencies - Complements #2383 (Romhacking / translation / localization reverse engineering) - Complements #3648 (Ren'Py reverse engineering: RPA archives, RPYC compiled scripts) - Complements #3647 (RPG Maker reverse engineering) - Complements #3632 (GameMaker reverse engineering)

bsakaguchi referenced this issue

2026-10-17 21:01:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:11:01 +00:00

[PR #1393] [MERGED] fix(javascript): stop reading media preload attributes as Electron preloads #2850