[GH-ISSUE #3677] [Feature] LLVM bitcode/LTO reverse engineering: .bc, embedded bitcode, ThinLTO summaries and IR graph reconstruction #1389

New issue

Open

opened 2026-10-16 21:05:00 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:05:00 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3677

Summary

LLVM Intermediate Representation (LLVM IR) and LLVM Bitcode (.bc files) form the universal compiler substrate for modern programming languages: Clang (C/C++), Rust (rustc), Swift, Kotlin/Native, Zig, and Julia. Furthermore, production binaries often bundle LLVM bitcode:

Apple macOS/iOS Mach-O binaries embedding __LLVM / __bundle sections for App Store bitcode submission;
Link-Time Optimization (LTO / ThinLTO) intermediate object archives (.a, .o);
GPU shaders and OpenCL kernels compiled to LLVM bitcode (e.g. Apple Metal AIR, AMD ROCm).

While REA has proposed native C++ (#3624) and shader (#3620) analysis, it currently treats standalone .bc files and embedded __LLVM sections as opaque binaries.

This feature proposal introduces dedicated LLVM Bitcode and LTO Reverse Engineering to REA, enabling automated bitstream decompression, parsing of LLVM IR blocks, extraction of functions/globals/metadata, ThinLTO summary index inspection, and call graph reconstruction.

Technical Capabilities & Architecture

                                    +-----------------------------------------+
                                    | Input: .bc file, Mach-O __bundle, or    |
                                    |        LTO object (.o) / archive (.a)   |
                                    +--------------------+--------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | Bitstream Wrapper &       |                                   | Section Carving           |
           | Magic Parser (BC / 0xDEC04342)                                | (Mach-O __LLVM, __bundle) |
           +-------------+-------------+                                   +-------------+-------------+
                         |                                                               |
                         +-------------------------------+-------------------------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | LLVM Bitcode Block & Record Decoder   |
                                     | (BlockInfo, Constants, Types, Globals)|
                                     +-------------------+-------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                               |                               |
                         v                               v                               v
           +---------------------------+   +---------------------------+   +---------------------------+
           | Function IR & Control     |   | ThinLTO Summary Index     |   | Debug Info & Metadata     |
           | Flow Graph (Basic Blocks) |   | (CallGraph, Cross-Module) |   | Reconstruction (!dbg, DIL)
           +-------------+-------------+   +-------------+-------------+   +-------------+-------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | REA Evidence: Reconstructed LLVM IR,  |
                                     | Function Call Graph & Source Maps     |
                                     +---------------------------------------+

1. Bitcode Container & Wrapper Ingestion

Wrapper Parsing: Detect and unpack bitcode wrappers:
- Standard bitcode magic (BC / 0xDEC04342).
- Bitcode wrapper header (0x0B17C0DE specifying offset, size, and CPU architecture).
- Mach-O segment carving: extract embedded bitcode bundles from __LLVM,__bundle or __LLVM,__bitcode sections.

2. LLVM Bitstream Block & Record Traversal

Bitstream Decoding: Implement an inert bitstream reader:
- Block header decoding: Block IDs (MODULE_BLOCK_ID, TYPE_BLOCK_ID_NEW, CONSTANTS_BLOCK_ID, FUNCTION_BLOCK_ID, METADATA_BLOCK_ID).
- VBR (Variable Bit Rate) and fixed-width integer decoding.
- Abbreviation table parsing (DEFINE_ABBREV).

3. Function IR & Control Flow Graph Reconstruction

Type & Global Recovery: Parse structural types (structs, arrays, pointers, function signatures) and global variable initializers.
Function Disassembly: Reconstruct basic blocks, SSA value registers, and instruction opcodes (alloca, load, store, getelementptr, call, br, switch, ret).
Control Flow Graph (CFG): Generate directed graphs of basic blocks, loop headers, and branching conditions.

4. ThinLTO Module Summary & Metadata Analysis

ThinLTO Index: Parse global value summaries, cross-module import candidates, and call-graph edge weights.
Debug Metadata Carving: Parse LLVM metadata nodes (!dbg, !DICompileUnit, !DISubprogram, !DILocalVariable), recovering original source filenames, directory paths, and source line numbers.

Acceptance Criteria

Introduce analyze_llvm_bitcode in REA CLI and MCP contracts, accepting .bc files and Mach-O binaries with embedded bitcode.
Ingest synthetic LLVM bitcode test files and decompress bitstream blocks without errors.
Reconstruct human-readable LLVM textual IR from bitcode function bodies.
Extract embedded DWARF/LLVM debug metadata to identify original source file paths and function lines.
Export function call graphs and ThinLTO cross-module summary links in structured JSON evidence.

#3620 [Feature] Shader and GPU Bytecode Reverse Engineering
#3624 [Feature] Native C++ semantic reconstruction
#3673 [Feature] ML compiler IR reverse engineering

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3677 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ## Summary LLVM Intermediate Representation (LLVM IR) and LLVM Bitcode (`.bc` files) form the universal compiler substrate for modern programming languages: Clang (C/C++), Rust (`rustc`), Swift, Kotlin/Native, Zig, and Julia. Furthermore, production binaries often bundle LLVM bitcode: - Apple macOS/iOS Mach-O binaries embedding `__LLVM` / `__bundle` sections for App Store bitcode submission; - Link-Time Optimization (LTO / ThinLTO) intermediate object archives (`.a`, `.o`); - GPU shaders and OpenCL kernels compiled to LLVM bitcode (e.g. Apple Metal AIR, AMD ROCm). While REA has proposed native C++ (#3624) and shader (#3620) analysis, it currently treats standalone `.bc` files and embedded `__LLVM` sections as opaque binaries. This feature proposal introduces dedicated **LLVM Bitcode and LTO Reverse Engineering** to REA, enabling automated bitstream decompression, parsing of LLVM IR blocks, extraction of functions/globals/metadata, ThinLTO summary index inspection, and call graph reconstruction. --- ## Technical Capabilities & Architecture ``` +-----------------------------------------+ | Input: .bc file, Mach-O __bundle, or | | LTO object (.o) / archive (.a) | +--------------------+--------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | Bitstream Wrapper & | | Section Carving | | Magic Parser (BC / 0xDEC04342) | (Mach-O __LLVM, __bundle) | +-------------+-------------+ +-------------+-------------+ | | +-------------------------------+-------------------------------+ | v +---------------------------------------+ | LLVM Bitcode Block & Record Decoder | | (BlockInfo, Constants, Types, Globals)| +-------------------+-------------------+ | +-------------------------------+-------------------------------+ | | | v v v +---------------------------+ +---------------------------+ +---------------------------+ | Function IR & Control | | ThinLTO Summary Index | | Debug Info & Metadata | | Flow Graph (Basic Blocks) | | (CallGraph, Cross-Module) | | Reconstruction (!dbg, DIL) +-------------+-------------+ +-------------+-------------+ +-------------+-------------+ | v +---------------------------------------+ | REA Evidence: Reconstructed LLVM IR, | | Function Call Graph & Source Maps | +---------------------------------------+ ``` ### 1. Bitcode Container & Wrapper Ingestion - **Wrapper Parsing:** Detect and unpack bitcode wrappers: - Standard bitcode magic (`BC` / `0xDEC04342`). - Bitcode wrapper header (`0x0B17C0DE` specifying offset, size, and CPU architecture). - Mach-O segment carving: extract embedded bitcode bundles from `__LLVM,__bundle` or `__LLVM,__bitcode` sections. ### 2. LLVM Bitstream Block & Record Traversal - **Bitstream Decoding:** Implement an inert bitstream reader: - Block header decoding: Block IDs (`MODULE_BLOCK_ID`, `TYPE_BLOCK_ID_NEW`, `CONSTANTS_BLOCK_ID`, `FUNCTION_BLOCK_ID`, `METADATA_BLOCK_ID`). - VBR (Variable Bit Rate) and fixed-width integer decoding. - Abbreviation table parsing (`DEFINE_ABBREV`). ### 3. Function IR & Control Flow Graph Reconstruction - **Type & Global Recovery:** Parse structural types (structs, arrays, pointers, function signatures) and global variable initializers. - **Function Disassembly:** Reconstruct basic blocks, SSA value registers, and instruction opcodes (`alloca`, `load`, `store`, `getelementptr`, `call`, `br`, `switch`, `ret`). - **Control Flow Graph (CFG):** Generate directed graphs of basic blocks, loop headers, and branching conditions. ### 4. ThinLTO Module Summary & Metadata Analysis - **ThinLTO Index:** Parse global value summaries, cross-module import candidates, and call-graph edge weights. - **Debug Metadata Carving:** Parse LLVM metadata nodes (`!dbg`, `!DICompileUnit`, `!DISubprogram`, `!DILocalVariable`), recovering original source filenames, directory paths, and source line numbers. --- ## Acceptance Criteria 1. Introduce `analyze_llvm_bitcode` in REA CLI and MCP contracts, accepting `.bc` files and Mach-O binaries with embedded bitcode. 2. Ingest synthetic LLVM bitcode test files and decompress bitstream blocks without errors. 3. Reconstruct human-readable LLVM textual IR from bitcode function bodies. 4. Extract embedded DWARF/LLVM debug metadata to identify original source file paths and function lines. 5. Export function call graphs and ThinLTO cross-module summary links in structured JSON evidence. --- ## Related - #3620 [Feature] Shader and GPU Bytecode Reverse Engineering - #3624 [Feature] Native C++ semantic reconstruction - #3673 [Feature] ML compiler IR reverse engineering

bsakaguchi referenced this issue

2026-10-16 21:05:48 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-10-16 21:05:49 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-10-17 21:01:19 +00:00

[GH-ISSUE #1389] [Design] Add rsleigh as a headless static-analysis provider candidate #483