[GH-ISSUE #3681] [Feature] Embedding-model reverse engineering: pooling, normalization, dimensions, quantization and semantic-pipeline reconstruction #1394

New issue

Open

opened 2026-10-16 21:05:03 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:05:03 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3681

Problem Statement & Context

Embedding and representation models (e.g., BGE, E5, ModernBERT, GTE, Nomic, ColBERT, CLIP text/vision encoders) serve as the semantic backbone for retrieval-augmented generation (RAG), vector search, semantic cache, and reranking pipelines. While general neural-network graph inspection (#3637) analyzes raw compute graphs and vector stores (#3667) index resultant vectors, reverse engineering an unknown or fine-tuned embedding model package currently lacks high-level semantic pipeline recovery.

Embedding models are defined not just by their transformer backbone, but critically by their inference pipeline:

Pooling mechanism: CLS token extraction, mean pooling across non-padded tokens, last-token pooling (decoder-based embeddings), or attention-weighted pooling.
Post-processing & normalization: L2 normalization, learned dense projection layers, and Matryoshka Representation Learning (MRL) dimension slices (e.g., truncating 1024-dim to 512, 256, 128).
Query/Passage prompt conditioning: Asymmetric retrieval prefixes (e.g. "Represent this sentence for searching relevant passages: " or "query: " vs "passage: ").
Late-interaction & multi-vector architectures: Token-level embeddings (ColBERT MaxSim, SPLADE sparse vectors).
Quantization & embedding compression: Binary/ubinary embeddings, scalar int8 quantization, and codebook product quantization.

Without dedicated embedding reverse engineering, analysts cannot reliably verify model dimension compatibility, replicate semantic search pipelines, audit retrieval biases, or detect custom dense projection adapters embedded within safetensors/GGUF checkpoints.

Proposed Architecture

flowchart TD
    A["Model Package / Safetensors / GGUF / ONNX"] --> B["Embedding Model Analyzer"]
    B --> C["Backbone & Head Extractor"]
    B --> D["Pipeline Configuration Extractor"]
    B --> E["Multi-Vector / Late-Interaction Inspector"]
    
    C --> C1["Transformer Layers & Hidden Dim"]
    C --> C2["Dense Projections & MRL Slices"]
    C --> C3["Quantized Weights (int8/fp8/binary)"]
    
    D --> D1["Pooling Strategy (Mean, CLS, Last-Token)"]
    D --> D2["Normalization Flags (L2 Unit Norm)"]
    D --> D3["Instruction / Prefix Templates"]
    
    E --> E1["ColBERT MaxSim Token Projections"]
    E --> E2["Sparse Lexical Weighting (SPLADE)"]
    
    C & D & E --> F["Synthesized Semantic Embedding Manifest & Pipeline Driver"]

Technical Capabilities & Modules

Backbone & Projection Head Discovery:
- Detect hidden dimension size (D_{model}) and output embedding dimension (D_{out}).
- Identify presence of dense linear projection heads (2_Dense, linear.weight), activation functions, and bias vectors.
- Detect Matryoshka Representation Learning (MRL) configurations and validate downstream performance across truncated sub-dimensions.
Pooling & Pipeline Reconstruction:
- Extract pooling strategies from modules.json, 1_Pooling/config.json, or infer from graph topology:
  - mean: Average pooling over non-padded tokens with attention mask weighting.
  - cls: First token [CLS] / <s> extraction.
  - last_token: Final token extraction (standard for causal decoder models like Mistral/Qwen embeddings).
  - weighted_mean: Position-weighted or decaying token aggregation.
- Verify normalize_embeddings boolean (L2 Euclidean normalization).
Retrieval Prefix & Instruction Recovery:
- Extract asymmetric search prefixes (prompt_name, query_prefix, passage_prefix) from configs or tokenizer pre-pended tokens.
- Correlate task-specific prompts for classification, clustering, semantic textual similarity (STS), and asymmetric retrieval.
Multi-Vector & Sparse Representation Analysis:
- Detect ColBERT-style late interaction models: token projection layers, vector compression codebooks, and MaxSim operator topologies.
- Detect SPLADE sparse neural representation heads (ReLU + log1p activation over full vocabulary).
Quantization & Codebook Recovery:
- Inspect static scalar quantization (int8/fp8 scales and zero-points) on embedding matrices.
- Identify 1-bit binary / unsigned binary (ubinary) sign thresholds for hamming-distance vector search.

Acceptance Criteria

Automatic identification of embedding model architecture (dense bi-encoder, MRL, late-interaction ColBERT, sparse SPLADE).
Accurate extraction and validation of pooling strategy, normalization, and output dimensions.
Extraction of asymmetric query/document prompt templates and tokenizer prefix requirements.
Verification of dense projection layers and MRL truncation configurations.
Export of standardized semantic embedding specification compatible with sentence-transformers, Hugging Face TEI, and ONNX Runtime.

Complements #3323 (AI model reverse engineering: GGUF, Safetensors, architecture)
Complements #3637 (ML model graph reverse engineering: ONNX, TFLite, CoreML)
Complements #3667 (RAG and vector-store reverse engineering)

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3681 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ### Problem Statement & Context Embedding and representation models (e.g., BGE, E5, ModernBERT, GTE, Nomic, ColBERT, CLIP text/vision encoders) serve as the semantic backbone for retrieval-augmented generation (RAG), vector search, semantic cache, and reranking pipelines. While general neural-network graph inspection (#3637) analyzes raw compute graphs and vector stores (#3667) index resultant vectors, reverse engineering an unknown or fine-tuned embedding model package currently lacks high-level semantic pipeline recovery. Embedding models are defined not just by their transformer backbone, but critically by their inference pipeline: 1. **Pooling mechanism**: CLS token extraction, mean pooling across non-padded tokens, last-token pooling (decoder-based embeddings), or attention-weighted pooling. 2. **Post-processing & normalization**: L2 normalization, learned dense projection layers, and Matryoshka Representation Learning (MRL) dimension slices (e.g., truncating 1024-dim to 512, 256, 128). 3. **Query/Passage prompt conditioning**: Asymmetric retrieval prefixes (e.g. `"Represent this sentence for searching relevant passages: "` or `"query: "` vs `"passage: "`). 4. **Late-interaction & multi-vector architectures**: Token-level embeddings (ColBERT MaxSim, SPLADE sparse vectors). 5. **Quantization & embedding compression**: Binary/ubinary embeddings, scalar int8 quantization, and codebook product quantization. Without dedicated embedding reverse engineering, analysts cannot reliably verify model dimension compatibility, replicate semantic search pipelines, audit retrieval biases, or detect custom dense projection adapters embedded within safetensors/GGUF checkpoints. ### Proposed Architecture ```mermaid flowchart TD A["Model Package / Safetensors / GGUF / ONNX"] --> B["Embedding Model Analyzer"] B --> C["Backbone & Head Extractor"] B --> D["Pipeline Configuration Extractor"] B --> E["Multi-Vector / Late-Interaction Inspector"] C --> C1["Transformer Layers & Hidden Dim"] C --> C2["Dense Projections & MRL Slices"] C --> C3["Quantized Weights (int8/fp8/binary)"] D --> D1["Pooling Strategy (Mean, CLS, Last-Token)"] D --> D2["Normalization Flags (L2 Unit Norm)"] D --> D3["Instruction / Prefix Templates"] E --> E1["ColBERT MaxSim Token Projections"] E --> E2["Sparse Lexical Weighting (SPLADE)"] C & D & E --> F["Synthesized Semantic Embedding Manifest & Pipeline Driver"] ``` ### Technical Capabilities & Modules 1. **Backbone & Projection Head Discovery:** - Detect hidden dimension size ($D_{model}$) and output embedding dimension ($D_{out}$). - Identify presence of dense linear projection heads (`2_Dense`, `linear.weight`), activation functions, and bias vectors. - Detect Matryoshka Representation Learning (MRL) configurations and validate downstream performance across truncated sub-dimensions. 2. **Pooling & Pipeline Reconstruction:** - Extract pooling strategies from `modules.json`, `1_Pooling/config.json`, or infer from graph topology: - `mean`: Average pooling over non-padded tokens with attention mask weighting. - `cls`: First token `[CLS]` / `<s>` extraction. - `last_token`: Final token extraction (standard for causal decoder models like Mistral/Qwen embeddings). - `weighted_mean`: Position-weighted or decaying token aggregation. - Verify `normalize_embeddings` boolean (L2 Euclidean normalization). 3. **Retrieval Prefix & Instruction Recovery:** - Extract asymmetric search prefixes (`prompt_name`, `query_prefix`, `passage_prefix`) from configs or tokenizer pre-pended tokens. - Correlate task-specific prompts for classification, clustering, semantic textual similarity (STS), and asymmetric retrieval. 4. **Multi-Vector & Sparse Representation Analysis:** - Detect ColBERT-style late interaction models: token projection layers, vector compression codebooks, and MaxSim operator topologies. - Detect SPLADE sparse neural representation heads (ReLU + log1p activation over full vocabulary). 5. **Quantization & Codebook Recovery:** - Inspect static scalar quantization (int8/fp8 scales and zero-points) on embedding matrices. - Identify 1-bit binary / unsigned binary (ubinary) sign thresholds for hamming-distance vector search. ### Acceptance Criteria - [ ] Automatic identification of embedding model architecture (dense bi-encoder, MRL, late-interaction ColBERT, sparse SPLADE). - [ ] Accurate extraction and validation of pooling strategy, normalization, and output dimensions. - [ ] Extraction of asymmetric query/document prompt templates and tokenizer prefix requirements. - [ ] Verification of dense projection layers and MRL truncation configurations. - [ ] Export of standardized semantic embedding specification compatible with sentence-transformers, Hugging Face TEI, and ONNX Runtime. ### Related Issues & Dependencies - Complements #3323 (AI model reverse engineering: GGUF, Safetensors, architecture) - Complements #3637 (ML model graph reverse engineering: ONNX, TFLite, CoreML) - Complements #3667 (RAG and vector-store reverse engineering)

bsakaguchi referenced this issue

2026-10-17 21:11:02 +00:00

[PR #1394] [MERGED] test(native): skip the EACCES directory diagnostic as root #2851

bsakaguchi referenced this issue

2026-10-17 21:12:36 +00:00

[PR #1584] [MERGED] test(native): skip the DMG EACCES diagnostic as root #2989