[GH-ISSUE #3620] [Feature] Shader and GPU Bytecode Reverse Engineering: SPIR-V, DXBC, DXIL, and Metal AIR reflection and decompilation #1349

New issue

Open

opened 2026-10-16 21:04:33 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:04:33 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3620

Summary

Shaders and GPU compute kernels represent a critical, often opaque component of modern software: games, 3D engines, computer vision pipelines, graphics drivers, machine learning accelerators, and UI composition systems. In production binaries and asset packages, high-level shader languages (HLSL, GLSL, Metal Shading Language, WGSL) are compiled into low-level intermediate representations and GPU bytecodes:

SPIR-V: Khronos standard portable intermediate representation for Vulkan, OpenCL, and WebGPU.
DXBC: DirectX 9/10/11 Shader Bytecode container format (DXBC).
DXIL: DirectX 12 Intermediate Language based on LLVM 3.7 bitcode with DirectX intrinsics.
Metal AIR (.metallib): Apple Metal Intermediate Representation containing LLVM bitcode for Apple Silicon GPUs.

Currently, REA does not cover shader binaries or GPU bytecodes. When encountered in game binaries, assets, or driver caches, these payloads remain unanalyzed blobs.

This feature proposal introduces dedicated Shader and GPU Bytecode Reverse Engineering to REA, enabling container parsing, shader reflection (resource bindings, constant buffers, input/output signatures), disassembly, high-level decompilation (HLSL/GLSL/WGSL), and rendering pipeline reconstruction.

Technical Capabilities

1. Shader Container & Format Ingestion

SPIR-V Binary Parser: Parse 32-bit SPIR-V binary modules (magic 0x07230203), extracting version, generator ID, bound ID, schema, and instruction streams (OpCapability, OpExtension, OpMemoryModel, OpEntryPoint, OpExecutionMode).
DirectX Bytecode (DXBC): Parse DXBC container headers, checksums, and chunk tables (RDEF Resource Definition, ISGN Input Signature, OSGN Output Signature, SHDR / SHEX Shader Code, STAT Statistics).
DirectX 12 Intermediate Language (DXIL): Extract DXIL / ILDB chunks and parse the embedded LLVM bitcode module, recognizing DirectX driver intrinsics, resource handles, and shader models (SM 6.0 through 6.8+).
Apple Metal Library (.metallib): Parse Metal library headers, function tables, and Apple LLVM bitcode (AIR) modules.

2. Shader Reflection & Interface Extraction

Resource Binding Map:
- Constant Buffers / Uniform Buffers (cbuffer / UBO): extract buffer names, slots, byte sizes, and member variable layouts with exact byte offsets.
- Shader Resource Views (SRV): textures (2D, 3D, Cube, Array), samplers, and storage buffers.
- Unordered Access Views (UAV): read/write storage buffers and images.
- Push Constants / Specialization Constants: extract offset ranges, default values, and constant IDs.
Input & Output Signatures:
- Recover vertex input layouts: semantic names (POSITION, NORMAL, TEXCOORD), register indices, component masks, and system values (SV_Position, SV_VertexID, SV_InstanceID).
- Recover pixel/fragment output targets: SV_Target0..N, SV_Depth, blend indices.
Compute Kernel Attributes: Extract thread group / workgroup dimensions (local_size_x, y, z), shared memory allocation sizes, and barrier synchronization points.

3. Disassembly & High-Level Decompilation

Disassembly: Emit clean, symbolic disassembly for DXBC assembly, DXIL LLVM IR, and SPIR-V human-readable assembly text.
High-Level Decompilation: Reconstruct high-level shader source code (HLSL, GLSL, or WGSL) from bytecode:
- Invert control flow graphs (loops, branches, switch statements, discards).
- Reconstruct mathematical vector/matrix operations (dot, cross, mul, swizzling .xyzw, normalize, saturate).
- Reconstruct texture sampling operations (Sample, SampleLevel, SampleGrad, texelFetch).

4. Rendering Pipeline & Visual Effect Reconstruction

Pipeline Pairing: Correlate complementary shader stages (Vertex + Pixel / Fragment, Hull + Domain / Tessellation, Compute) into coherent graphics pipeline states.
Lighting & Material Model Recovery: Identify standard shading models (PBR / Cook-Torrance, Blinn-Phong, Subsurface Scattering, Volumetric Fog) and post-processing passes (SSAO, Bloom, Tone Mapping, TAA).

Acceptance Criteria

Introduce analyze_shader_artifact in REA CLI and MCP contracts, accepting SPIR-V (.spv), DXBC/DXIL (.cso, .bin), and .metallib files.
Accurately extract shader stage (Vertex, Pixel/Fragment, Compute, Geometry, Mesh, RayTracing) and shader model version.
Output complete reflection metadata (constant buffer layouts, texture bindings, input/output registers) in structured JSON evidence.
Decompile synthetic SPIR-V and DXBC test shaders into valid HLSL/GLSL code.
Enforce bounded parsing and safe error handling for malformed or corrupted shader blobs.

#743 [Feature] Recover .NET NativeAOT metadata in Ghidra workflows
#3607 [Feature] Comprehensive Unity extraction and reverse engineering
#3609 [Feature] Video Reverse Engineering

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3620 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: AI & Runtimes](https://img.shields.io/badge/domain-AI_&_runtimes-blueviolet.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ## Summary Shaders and GPU compute kernels represent a critical, often opaque component of modern software: games, 3D engines, computer vision pipelines, graphics drivers, machine learning accelerators, and UI composition systems. In production binaries and asset packages, high-level shader languages (HLSL, GLSL, Metal Shading Language, WGSL) are compiled into low-level intermediate representations and GPU bytecodes: - **SPIR-V:** Khronos standard portable intermediate representation for Vulkan, OpenCL, and WebGPU. - **DXBC:** DirectX 9/10/11 Shader Bytecode container format (`DXBC`). - **DXIL:** DirectX 12 Intermediate Language based on LLVM 3.7 bitcode with DirectX intrinsics. - **Metal AIR (.metallib):** Apple Metal Intermediate Representation containing LLVM bitcode for Apple Silicon GPUs. Currently, REA does not cover shader binaries or GPU bytecodes. When encountered in game binaries, assets, or driver caches, these payloads remain unanalyzed blobs. This feature proposal introduces dedicated **Shader and GPU Bytecode Reverse Engineering** to REA, enabling container parsing, shader reflection (resource bindings, constant buffers, input/output signatures), disassembly, high-level decompilation (HLSL/GLSL/WGSL), and rendering pipeline reconstruction. --- ## Technical Capabilities ### 1. Shader Container & Format Ingestion - **SPIR-V Binary Parser:** Parse 32-bit SPIR-V binary modules (magic `0x07230203`), extracting version, generator ID, bound ID, schema, and instruction streams (`OpCapability`, `OpExtension`, `OpMemoryModel`, `OpEntryPoint`, `OpExecutionMode`). - **DirectX Bytecode (DXBC):** Parse `DXBC` container headers, checksums, and chunk tables (`RDEF` Resource Definition, `ISGN` Input Signature, `OSGN` Output Signature, `SHDR` / `SHEX` Shader Code, `STAT` Statistics). - **DirectX 12 Intermediate Language (DXIL):** Extract `DXIL` / `ILDB` chunks and parse the embedded LLVM bitcode module, recognizing DirectX driver intrinsics, resource handles, and shader models (SM 6.0 through 6.8+). - **Apple Metal Library (.metallib):** Parse Metal library headers, function tables, and Apple LLVM bitcode (AIR) modules. ### 2. Shader Reflection & Interface Extraction - **Resource Binding Map:** - Constant Buffers / Uniform Buffers (`cbuffer` / UBO): extract buffer names, slots, byte sizes, and member variable layouts with exact byte offsets. - Shader Resource Views (SRV): textures (2D, 3D, Cube, Array), samplers, and storage buffers. - Unordered Access Views (UAV): read/write storage buffers and images. - Push Constants / Specialization Constants: extract offset ranges, default values, and constant IDs. - **Input & Output Signatures:** - Recover vertex input layouts: semantic names (`POSITION`, `NORMAL`, `TEXCOORD`), register indices, component masks, and system values (`SV_Position`, `SV_VertexID`, `SV_InstanceID`). - Recover pixel/fragment output targets: `SV_Target0`..`N`, `SV_Depth`, blend indices. - **Compute Kernel Attributes:** Extract thread group / workgroup dimensions (`local_size_x, y, z`), shared memory allocation sizes, and barrier synchronization points. ### 3. Disassembly & High-Level Decompilation - **Disassembly:** Emit clean, symbolic disassembly for DXBC assembly, DXIL LLVM IR, and SPIR-V human-readable assembly text. - **High-Level Decompilation:** Reconstruct high-level shader source code (HLSL, GLSL, or WGSL) from bytecode: - Invert control flow graphs (loops, branches, switch statements, discards). - Reconstruct mathematical vector/matrix operations (`dot`, `cross`, `mul`, swizzling `.xyzw`, `normalize`, `saturate`). - Reconstruct texture sampling operations (`Sample`, `SampleLevel`, `SampleGrad`, `texelFetch`). ### 4. Rendering Pipeline & Visual Effect Reconstruction - **Pipeline Pairing:** Correlate complementary shader stages (Vertex + Pixel / Fragment, Hull + Domain / Tessellation, Compute) into coherent graphics pipeline states. - **Lighting & Material Model Recovery:** Identify standard shading models (PBR / Cook-Torrance, Blinn-Phong, Subsurface Scattering, Volumetric Fog) and post-processing passes (SSAO, Bloom, Tone Mapping, TAA). --- ## Acceptance Criteria 1. Introduce `analyze_shader_artifact` in REA CLI and MCP contracts, accepting SPIR-V (`.spv`), DXBC/DXIL (`.cso`, `.bin`), and `.metallib` files. 2. Accurately extract shader stage (Vertex, Pixel/Fragment, Compute, Geometry, Mesh, RayTracing) and shader model version. 3. Output complete reflection metadata (constant buffer layouts, texture bindings, input/output registers) in structured JSON evidence. 4. Decompile synthetic SPIR-V and DXBC test shaders into valid HLSL/GLSL code. 5. Enforce bounded parsing and safe error handling for malformed or corrupted shader blobs. --- ## Related - #743 [Feature] Recover .NET NativeAOT metadata in Ghidra workflows - #3607 [Feature] Comprehensive Unity extraction and reverse engineering - #3609 [Feature] Video Reverse Engineering

bsakaguchi referenced this issue

2026-10-17 21:10:37 +00:00

[PR #1349] [MERGED] docs: celebrate 50,000 GitHub stars 🎉 #2815

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1349

No description provided.

Rows
Columns