[GH-ISSUE #3722] [Feature] On-device AI runtime reverse engineering: Core AI, Foundation Models, ExecuTorch, LiteRT and NPU delegate reconstruction #1434

New issue

Open

opened 2026-10-16 21:05:23 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:05:23 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3722

Problem Statement & Context

Operating systems and mobile/embedded platforms are aggressively shifting AI inference from remote cloud APIs to local on-device hardware accelerators (Neural Processing Units / NPUs):

Apple Core AI & System Foundation Models: Frameworks for running local 3B+ parameter models on Apple Silicon Neural Engine (NPE/ANE), distributing models in .aimodel packages, adapter binaries, and CoreML runtime graphs.
PyTorch ExecuTorch: PyTorch's standardized on-device runtime targeting mobile, IoT, wearables, and AR/VR, deploying ahead-of-time (AOT) exported .pte program packages.
Google LiteRT (formerly TensorFlow Lite): High-performance on-device AI runtime utilizing hardware delegates (Hexagon NPU, MediaTek APU, GPU OpenCL/Vulkan).
Hardware Acceleration Delegates: Proprietary NPU compilation pipelines compiling quantized operators into proprietary microcode binaries.

When analyzing modern mobile or edge software, security analysts and reverse engineers encounter opaque binary model files (.pte, .aimodel, .tflite) linked to native application shells. REA needs a dedicated on-device AI analyzer capable of decompiling model packages, identifying hardware delegate offloads, and mapping the application-to-inference call graph.

Proposed Architecture

flowchart TD
    A["Application Package (iOS IPA, Android APK, Edge Firmware)"] --> B["On-Device AI Engine"]
    B --> C["Model Package Ingestion (.pte, .aimodel, .tflite)"]
    B --> D["Operator Graph & Quantization Inspector"]
    B --> E["Hardware Delegate & NPU Boundary Analyzer"]
    B --> F["Native App Inference Call Site Mapper"]
    
    C --> C1["ExecuTorch .pte Header & Program Schema"]
    C --> C2["Core AI .aimodel / CoreML Package Manifest"]
    
    D --> D1["Execution Plan & Operator Topology"]
    D --> D2["Quantization Calibration (INT4, INT8, FP16)"]
    
    E --> E1["NPU Delegate Signatures (Apple NPE, Qualcomm QNN, MediaTek)"]
    E --> E2["Custom Backend Offload Partitions"]
    
    F --> F1["App Entrypoints (Prompt Generation, Audio/Vision Preprocessing)"]
    F --> F2["Inference Result Decoding & Action Dispatch"]
    
    C & D & E & F --> G["Synthesized On-Device AI Architecture & Native Execution Graph"]

Technical Capabilities & Modules

Model Package Parsing (.pte, .aimodel, .tflite):
- Parse ExecuTorch .pte FlatBuffers containers: extract Program metadata, method headers (forward), execution plans, and tensor storages.
- Parse Apple .aimodel and compiled CoreML packages: model parameters, context window limits, and tokenizer configurations.
- Parse modern LiteRT / TFLite FlatBuffers schemas.
Operator Topology & Execution Graph Recovery:
- Map neural network operator sequences, subgraphs, control flow (loops, conditionals), and custom kernel registrations.
- Detect activation functions, normalization layers, and attention mechanisms.
Quantization & Weight Inspection:
- Decode mixed-precision quantization: 4-bit integer (INT4), 8-bit integer (INT8), bfloat16, and fp16 weights.
- Extract quantization scales, zero-points, and group dimensions.
Hardware Delegate & Acceleration Mapping:
- Detect backend delegation partitions: identify which graph operators execute on CPU fallback vs GPU vs dedicated NPU (Apple Neural Engine, Qualcomm QNN, Samsung NPU).
- Flag un-accelerated fallback bottlenecks.
App-to-Inference Call Graph Correlation:
- Locate native C++, Swift, or Kotlin call sites initializing the on-device AI runtime: model loading, buffer allocations, tensor input binding.
- Correlate application sensory inputs (camera frames, microphone audio, text queries) to model input tensors and subsequent output action triggers.

Acceptance Criteria

Automated detection and deserialization of .pte (ExecuTorch), .aimodel, and LiteRT model packages.
Structural recovery of operator topologies, tensor shapes, and data types.
Identification of quantization schemas and per-tensor calibration parameters.
Mapping of hardware delegate partitions across CPU, GPU, and NPU backends.
End-to-end correlation linking application code call sites to model execution entrypoints.

Complements #3637 (ML model graph reverse engineering: ONNX, TFLite, CoreML)
Complements #3668 (Local LLM runtime reverse engineering: llama.cpp, ExecuTorch)

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3722 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ### Problem Statement & Context Operating systems and mobile/embedded platforms are aggressively shifting AI inference from remote cloud APIs to local on-device hardware accelerators (Neural Processing Units / NPUs): - **Apple Core AI & System Foundation Models**: Frameworks for running local 3B+ parameter models on Apple Silicon Neural Engine (NPE/ANE), distributing models in `.aimodel` packages, adapter binaries, and CoreML runtime graphs. - **PyTorch ExecuTorch**: PyTorch's standardized on-device runtime targeting mobile, IoT, wearables, and AR/VR, deploying ahead-of-time (AOT) exported `.pte` program packages. - **Google LiteRT (formerly TensorFlow Lite)**: High-performance on-device AI runtime utilizing hardware delegates (Hexagon NPU, MediaTek APU, GPU OpenCL/Vulkan). - **Hardware Acceleration Delegates**: Proprietary NPU compilation pipelines compiling quantized operators into proprietary microcode binaries. When analyzing modern mobile or edge software, security analysts and reverse engineers encounter opaque binary model files (`.pte`, `.aimodel`, `.tflite`) linked to native application shells. REA needs a dedicated on-device AI analyzer capable of decompiling model packages, identifying hardware delegate offloads, and mapping the application-to-inference call graph. ### Proposed Architecture ```mermaid flowchart TD A["Application Package (iOS IPA, Android APK, Edge Firmware)"] --> B["On-Device AI Engine"] B --> C["Model Package Ingestion (.pte, .aimodel, .tflite)"] B --> D["Operator Graph & Quantization Inspector"] B --> E["Hardware Delegate & NPU Boundary Analyzer"] B --> F["Native App Inference Call Site Mapper"] C --> C1["ExecuTorch .pte Header & Program Schema"] C --> C2["Core AI .aimodel / CoreML Package Manifest"] D --> D1["Execution Plan & Operator Topology"] D --> D2["Quantization Calibration (INT4, INT8, FP16)"] E --> E1["NPU Delegate Signatures (Apple NPE, Qualcomm QNN, MediaTek)"] E --> E2["Custom Backend Offload Partitions"] F --> F1["App Entrypoints (Prompt Generation, Audio/Vision Preprocessing)"] F --> F2["Inference Result Decoding & Action Dispatch"] C & D & E & F --> G["Synthesized On-Device AI Architecture & Native Execution Graph"] ``` ### Technical Capabilities & Modules 1. **Model Package Parsing (`.pte`, `.aimodel`, `.tflite`):** - Parse ExecuTorch `.pte` FlatBuffers containers: extract Program metadata, method headers (`forward`), execution plans, and tensor storages. - Parse Apple `.aimodel` and compiled CoreML packages: model parameters, context window limits, and tokenizer configurations. - Parse modern LiteRT / TFLite FlatBuffers schemas. 2. **Operator Topology & Execution Graph Recovery:** - Map neural network operator sequences, subgraphs, control flow (loops, conditionals), and custom kernel registrations. - Detect activation functions, normalization layers, and attention mechanisms. 3. **Quantization & Weight Inspection:** - Decode mixed-precision quantization: 4-bit integer (INT4), 8-bit integer (INT8), bfloat16, and fp16 weights. - Extract quantization scales, zero-points, and group dimensions. 4. **Hardware Delegate & Acceleration Mapping:** - Detect backend delegation partitions: identify which graph operators execute on CPU fallback vs GPU vs dedicated NPU (Apple Neural Engine, Qualcomm QNN, Samsung NPU). - Flag un-accelerated fallback bottlenecks. 5. **App-to-Inference Call Graph Correlation:** - Locate native C++, Swift, or Kotlin call sites initializing the on-device AI runtime: model loading, buffer allocations, tensor input binding. - Correlate application sensory inputs (camera frames, microphone audio, text queries) to model input tensors and subsequent output action triggers. ### Acceptance Criteria - [ ] Automated detection and deserialization of `.pte` (ExecuTorch), `.aimodel`, and LiteRT model packages. - [ ] Structural recovery of operator topologies, tensor shapes, and data types. - [ ] Identification of quantization schemas and per-tensor calibration parameters. - [ ] Mapping of hardware delegate partitions across CPU, GPU, and NPU backends. - [ ] End-to-end correlation linking application code call sites to model execution entrypoints. ### Related Issues & Dependencies - Complements #3637 (ML model graph reverse engineering: ONNX, TFLite, CoreML) - Complements #3668 (Local LLM runtime reverse engineering: llama.cpp, ExecuTorch)

bsakaguchi referenced this issue

2026-10-17 21:01:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:11:23 +00:00

[PR #1434] [MERGED] perf(javascript): index change-graph nodes once per comparison #2884

bsakaguchi referenced this issue

2026-10-17 21:11:30 +00:00

[PR #1448] [MERGED] test(javascript): reuse comparison node fixtures #2894

bsakaguchi referenced this issue

2026-10-17 21:11:36 +00:00

[PR #1458] [MERGED] perf(javascript): derive the application graph ID once #2900