morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #3669] [Feature] Model lineage reverse engineering: base-model fingerprinting, shard topology, adapter ancestry, quantization provenance and model diff #1381

New issue

Open

opened 2026-10-16 21:04:54 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:04:54 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3669

Summary

The open-source AI ecosystem hosts hundreds of thousands of fine-tuned, merged, quantized, and adapted models (e.g. on Hugging Face and model hubs). In AI forensics, copyright compliance, model supply-chain audits, and vulnerability research, analysts encounter models whose origins are unverified, obfuscated, or mislabeled.

An analyst inspecting a model package needs to determine:

Which foundation base model was this trained or fine-tuned on (e.g. LLaMA 3, Mistral, Qwen 2.5)?
Is this model a merge of multiple models (e.g. MergeKit linear, spherical SLERP, DARE, ties)?
What adapter ancestry or LoRA fine-tuning was applied?
What is the shard topology and parameter integrity across multi-part checkpoints?
How does this model diff against a known-good reference model?

Currently, REA has no tooling to compare neural network models or track model provenance.

This feature proposal introduces dedicated Model Lineage and Model Diff Reverse Engineering to REA, enabling fingerprinting of base model architectures, shard index validation, adapter ancestry tracing, quantization provenance tracking, and structural tensor-level model diffing.

Technical Capabilities & Architecture

                                    +-----------------------------------------+
                                    | Input: Model A (Target) & Model B (Ref) |
                                    |        (safetensors, GGUF, config.json) |
                                    +--------------------+--------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | Shard Index & Topology    |                                   | Base Model Architectural  |
           | Validator (model.index)   |                                   | Fingerprinting            |
           +-------------+-------------+                                   +-------------+-------------+
                         |                                                               |
                         +-------------------------------+-------------------------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | Tensor Name & Shape Alignment Engine  |
                                     | (Exact matching, prefix normalization)|
                                     +-------------------+-------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | Model Diff & Deviation    |                                   | Model Merge & Adapter     |
           | Analysis (Tensor Hashes)  |                                   | Ancestry Reconstruction   |
           +-------------+-------------+                                   +-------------+-------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | REA Evidence: Lineage Dossier,        |
                                     | Ancestry Tree & Model Diff Report     |
                                     +---------------------------------------+

1. Shard Topology & Weight Integrity Validation

Shard Index Parser (model.safetensors.index.json):
- Parse weight_map dictionaries mapping tensor names to shard filenames (e.g. model-00001-of-00004.safetensors).
- Verify completeness: detect missing shards, orphan weight files, and duplicate tensor declarations.
- Validate total parameter count by aggregating declared shard shapes.

2. Base Model Architectural Fingerprinting

Structural Invariant Matching: Fingerprint target models by matching structural invariants against canonical architecture profiles:
- Layer count, attention heads (n_{heads}), key-value heads (n_{kv\_heads} - Grouped-Query Attention ratio).
- Hidden dimensions (d_{model}), intermediate MLP dimensions, and rotary embedding ( ext{RoPE}) base frequencies (e.g. \theta = 500000.0).
- Normalization layer types (RMSNorm vs. LayerNorm, \epsilon value).
- Vocabulary size matching known base tokenizers (e.g. 32000 for LLaMA 2, 128256 for LLaMA 3, 151936 for Qwen 2.5).

3. Tensor Name & Shape Alignment

Canonical Tensor Renaming: Normalize vendor-specific naming variations (e.g. model.layers.0.self_attn.q_proj.weight vs. transformer.h.0.attn.q_proj.weight).
Shape Alignment: Align corresponding tensors between target and reference models to verify layer-for-layer architectural compatibility.

4. Tensor-Level Model Diffing & Deviation Analysis

Digest-Based Exact Matching: Compare SHA-256 digests of identical tensors without decompressing weight arrays to identify unchanged layers (e.g. identical embedding layers or frozen lower transformer blocks).
Quantization Provenance: Trace whether a GGUF model was quantized from an official full-precision FP16 source or re-quantized from an already lossy quantization (double-quantization artifacting).

5. Model Merge & Adapter Ancestry Inversion

Merge Recipe Detection: Identify indicators of model merging techniques (MergeKit):
- Detect SLERP (Spherical Linear Interpolation) or TIES/DARE sparsification artifacts across weight matrices.
- Extract embedded merge configurations in config.json or model card metadata.
Lineage Tree Reconstruction: Construct a visual ancestry tree:
Foundation Base Model -> Fine-Tuning Stage -> LoRA Adapter Fusion -> Model Merge -> Final Quantization.

Acceptance Criteria

Introduce diff_models and fingerprint_model_lineage in REA CLI and MCP contracts.
Validate multi-part shard indices (model.safetensors.index.json) and report shard completeness.
Automatically identify base model architecture families (e.g. LLaMA, Mistral, Qwen) from structural tensor invariants.
Output a tensor-level diff between two synthetic model fixtures, classifying layers as identical, modified, or added.
Export model lineage trees and detected merge algorithms in structured JSON evidence.

#3323 [Feature] Inspect GGUF and safetensors model files without loading them
#3637 [Feature] ML model graph reverse engineering
#3652 [Feature] PyTorch checkpoint reverse engineering

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3669 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: AI & Runtimes](https://img.shields.io/badge/domain-AI_&_runtimes-blueviolet.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ## Summary The open-source AI ecosystem hosts hundreds of thousands of fine-tuned, merged, quantized, and adapted models (e.g. on Hugging Face and model hubs). In AI forensics, copyright compliance, model supply-chain audits, and vulnerability research, analysts encounter models whose origins are unverified, obfuscated, or mislabeled. An analyst inspecting a model package needs to determine: - Which foundation base model was this trained or fine-tuned on (e.g. LLaMA 3, Mistral, Qwen 2.5)? - Is this model a merge of multiple models (e.g. MergeKit linear, spherical SLERP, DARE, ties)? - What adapter ancestry or LoRA fine-tuning was applied? - What is the shard topology and parameter integrity across multi-part checkpoints? - How does this model diff against a known-good reference model? Currently, REA has no tooling to compare neural network models or track model provenance. This feature proposal introduces dedicated **Model Lineage and Model Diff Reverse Engineering** to REA, enabling fingerprinting of base model architectures, shard index validation, adapter ancestry tracing, quantization provenance tracking, and structural tensor-level model diffing. --- ## Technical Capabilities & Architecture ``` +-----------------------------------------+ | Input: Model A (Target) & Model B (Ref) | | (safetensors, GGUF, config.json) | +--------------------+--------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | Shard Index & Topology | | Base Model Architectural | | Validator (model.index) | | Fingerprinting | +-------------+-------------+ +-------------+-------------+ | | +-------------------------------+-------------------------------+ | v +---------------------------------------+ | Tensor Name & Shape Alignment Engine | | (Exact matching, prefix normalization)| +-------------------+-------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | Model Diff & Deviation | | Model Merge & Adapter | | Analysis (Tensor Hashes) | | Ancestry Reconstruction | +-------------+-------------+ +-------------+-------------+ | v +---------------------------------------+ | REA Evidence: Lineage Dossier, | | Ancestry Tree & Model Diff Report | +---------------------------------------+ ``` ### 1. Shard Topology & Weight Integrity Validation - **Shard Index Parser (`model.safetensors.index.json`):** - Parse `weight_map` dictionaries mapping tensor names to shard filenames (e.g. `model-00001-of-00004.safetensors`). - Verify completeness: detect missing shards, orphan weight files, and duplicate tensor declarations. - Validate total parameter count by aggregating declared shard shapes. ### 2. Base Model Architectural Fingerprinting - **Structural Invariant Matching:** Fingerprint target models by matching structural invariants against canonical architecture profiles: - Layer count, attention heads ($n_{heads}$), key-value heads ($n_{kv\_heads}$ - Grouped-Query Attention ratio). - Hidden dimensions ($d_{model}$), intermediate MLP dimensions, and rotary embedding ($ ext{RoPE}$) base frequencies (e.g. $\theta = 500000.0$). - Normalization layer types (RMSNorm vs. LayerNorm, $\epsilon$ value). - Vocabulary size matching known base tokenizers (e.g. 32000 for LLaMA 2, 128256 for LLaMA 3, 151936 for Qwen 2.5). ### 3. Tensor Name & Shape Alignment - **Canonical Tensor Renaming:** Normalize vendor-specific naming variations (e.g. `model.layers.0.self_attn.q_proj.weight` vs. `transformer.h.0.attn.q_proj.weight`). - **Shape Alignment:** Align corresponding tensors between target and reference models to verify layer-for-layer architectural compatibility. ### 4. Tensor-Level Model Diffing & Deviation Analysis - **Digest-Based Exact Matching:** Compare SHA-256 digests of identical tensors without decompressing weight arrays to identify unchanged layers (e.g. identical embedding layers or frozen lower transformer blocks). - **Quantization Provenance:** Trace whether a GGUF model was quantized from an official full-precision FP16 source or re-quantized from an already lossy quantization (double-quantization artifacting). ### 5. Model Merge & Adapter Ancestry Inversion - **Merge Recipe Detection:** Identify indicators of model merging techniques (MergeKit): - Detect SLERP (Spherical Linear Interpolation) or TIES/DARE sparsification artifacts across weight matrices. - Extract embedded merge configurations in `config.json` or model card metadata. - **Lineage Tree Reconstruction:** Construct a visual ancestry tree: `Foundation Base Model -> Fine-Tuning Stage -> LoRA Adapter Fusion -> Model Merge -> Final Quantization`. --- ## Acceptance Criteria 1. Introduce `diff_models` and `fingerprint_model_lineage` in REA CLI and MCP contracts. 2. Validate multi-part shard indices (`model.safetensors.index.json`) and report shard completeness. 3. Automatically identify base model architecture families (e.g. LLaMA, Mistral, Qwen) from structural tensor invariants. 4. Output a tensor-level diff between two synthetic model fixtures, classifying layers as identical, modified, or added. 5. Export model lineage trees and detected merge algorithms in structured JSON evidence. --- ## Related - #3323 [Feature] Inspect GGUF and safetensors model files without loading them - #3637 [Feature] ML model graph reverse engineering - #3652 [Feature] PyTorch checkpoint reverse engineering

bsakaguchi referenced this issue

2026-10-17 21:10:55 +00:00

[PR #1381] [MERGED] chore(release/6.3.0): release rea-agents 6.3.0 #2841

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1381

No description provided.

Rows
Columns