alpha, target modules and merge-plan reconstruction #1376

New issue

Open

opened 2026-10-16 21:04:50 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:04:50 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3665

Summary

Low-Rank Adaptation (LoRA) and Parameter-Efficient Fine-Tuning (PEFT) are the standard techniques for fine-tuning Large Language Models, diffusion models, and multimodal foundation models. Rather than distributing full multi-gigabyte model weights, developers distribute lightweight LoRA adapters (adapter_config.json + adapter_model.safetensors or .bin), injecting low-rank matrix pairs (W = W_0 + \frac{\alpha}{r} B \cdot A) into attention and feed-forward layers.

While issue #3323 covers basic safetensors header reading, it does not interpret PEFT adapter semantics or determine how an adapter integrates into its foundation model.

This feature proposal introduces dedicated LoRA and PEFT Adapter Reverse Engineering to REA, enabling automated inspection of adapter architectures, identification of targeted attention/MLP layers, rank/alpha scaling factor recovery, DoRA (Weight-Decomposed Low-Rank Adaptation) verification, and computation of exact weight merge plans into base models.

Technical Capabilities & Architecture

                                    +-----------------------------------------+
                                    | Input: LoRA Adapter Directory or Files  |
                                    | (adapter_config.json, adapter_model)    |
                                    +--------------------+--------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | Adapter Config Parser     |                                   | Low-Rank Tensor Storage   |
           | (PEFT Type, Rank, Alpha)  |                                   | (lora_A, lora_B, Safetens)|
           +-------------+-------------+                                   +-------------+-------------+
                         |                                                               |
                         +-------------------------------+-------------------------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | Target Layer & Module Mapping         |
                                     | (q_proj, v_proj, gate_up, down_proj)  |
                                     +-------------------+-------------------+
                                                         |
                         +-------------------------------+-------------------------------+
                         |                                                               |
                         v                                                               v
           +---------------------------+                                   +---------------------------+
           | Base Model Compatibility  |                                   | Merge-Plan Synthesizer    |
           | Verification (Shape Match)|                                   | (Delta W = alpha/r * B*A) |
           +-------------+-------------+                                   +-------------+-------------+
                                                         |
                                                         v
                                     +---------------------------------------+
                                     | REA Evidence: Adapter Architecture,   |
                                     | Layer Offsets & Model Merge Plan      |
                                     +---------------------------------------+

1. Adapter Configuration & Metadata Ingestion

Config Parser (adapter_config.json): Parse PEFT configuration metadata:
- peft_type: LoRA, DoRA, AdaLoRA, Prefix Tuning, Prompt Tuning, IA3.
- r (Rank dimension) and lora_alpha (Scaling hyperparameter).
- lora_dropout: Dropout probability applied during training.
- target_modules: Targeted module names (e.g. q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).
- modules_to_save: Non-LoRA modules trained alongside (e.g. embed_tokens, lm_head).
- base_model_name_or_path: Identifier of the foundation model.

2. Low-Rank Tensor Decomposition

Tensor Matrix Pairing: Extract and pair low-rank matrices from adapter_model.safetensors:
- lora_A down-projection weights with dimensions [r, d_{in}].
- lora_B up-projection weights with dimensions [d_{out}, r].
- Calculate matrix inner product dimensions and effective rank constraints.
DoRA Magnitude Vectors: For Weight-Decomposed LoRA (DoRA), extract directional low-rank tensors and companion magnitude scaling vectors (lora_magnitude_vector).

3. Base Model Binding & Compatibility Verification

Layer Stride Verification: Match adapter module paths (e.g. base_model.model.model.layers.31.self_attn.q_proj) against target base model architectures (LLaMA, Mistral, Gemma, Qwen, DeepSeek).
Dimension Compatibility: Verify whether d_{in} and d_{out} match the base model's hidden dimension (d_{model}) and intermediate MLP dimensions.

4. Merge-Plan Reconstruction

Mathematical Scaling Inversion: Reconstruct the mathematical merge formula for each targeted layer:
```
\Delta W = \left( \frac{\alpha}{r} \right) (B \cdot A)
```
Memory & Precision Footprint: Calculate the memory and precision requirements necessary to fuse the adapter into base model weights without running PyTorch or Python.

Acceptance Criteria

Introduce analyze_lora_adapter in REA CLI and MCP contracts, accepting adapter directories or files.
Accurately parse adapter_config.json across LoRA and DoRA synthetic test fixtures.
Pair lora_A and lora_B tensor arrays and verify inner rank dimension r.
Validate base model compatibility and report target layer coverage in structured evidence.
Compute the mathematical merge scaling factor and output the layer-by-layer merge plan.

#3323 [Feature] Inspect GGUF and safetensors model files without loading them
#3651 [Feature] LLM application reverse engineering
#3652 [Feature] PyTorch checkpoint reverse engineering

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3665 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ## Summary Low-Rank Adaptation (LoRA) and Parameter-Efficient Fine-Tuning (PEFT) are the standard techniques for fine-tuning Large Language Models, diffusion models, and multimodal foundation models. Rather than distributing full multi-gigabyte model weights, developers distribute lightweight LoRA adapters (`adapter_config.json` + `adapter_model.safetensors` or `.bin`), injecting low-rank matrix pairs ($W = W_0 + \frac{\alpha}{r} B \cdot A$) into attention and feed-forward layers. While issue #3323 covers basic safetensors header reading, it does not interpret PEFT adapter semantics or determine how an adapter integrates into its foundation model. This feature proposal introduces dedicated **LoRA and PEFT Adapter Reverse Engineering** to REA, enabling automated inspection of adapter architectures, identification of targeted attention/MLP layers, rank/alpha scaling factor recovery, DoRA (Weight-Decomposed Low-Rank Adaptation) verification, and computation of exact weight merge plans into base models. --- ## Technical Capabilities & Architecture ``` +-----------------------------------------+ | Input: LoRA Adapter Directory or Files | | (adapter_config.json, adapter_model) | +--------------------+--------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | Adapter Config Parser | | Low-Rank Tensor Storage | | (PEFT Type, Rank, Alpha) | | (lora_A, lora_B, Safetens)| +-------------+-------------+ +-------------+-------------+ | | +-------------------------------+-------------------------------+ | v +---------------------------------------+ | Target Layer & Module Mapping | | (q_proj, v_proj, gate_up, down_proj) | +-------------------+-------------------+ | +-------------------------------+-------------------------------+ | | v v +---------------------------+ +---------------------------+ | Base Model Compatibility | | Merge-Plan Synthesizer | | Verification (Shape Match)| | (Delta W = alpha/r * B*A) | +-------------+-------------+ +-------------+-------------+ | v +---------------------------------------+ | REA Evidence: Adapter Architecture, | | Layer Offsets & Model Merge Plan | +---------------------------------------+ ``` ### 1. Adapter Configuration & Metadata Ingestion - **Config Parser (`adapter_config.json`):** Parse PEFT configuration metadata: - `peft_type`: LoRA, DoRA, AdaLoRA, Prefix Tuning, Prompt Tuning, IA3. - `r` (Rank dimension) and `lora_alpha` (Scaling hyperparameter). - `lora_dropout`: Dropout probability applied during training. - `target_modules`: Targeted module names (e.g. `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`). - `modules_to_save`: Non-LoRA modules trained alongside (e.g. `embed_tokens`, `lm_head`). - `base_model_name_or_path`: Identifier of the foundation model. ### 2. Low-Rank Tensor Decomposition - **Tensor Matrix Pairing:** Extract and pair low-rank matrices from `adapter_model.safetensors`: - `lora_A` down-projection weights with dimensions $[r, d_{in}]$. - `lora_B` up-projection weights with dimensions $[d_{out}, r]$. - Calculate matrix inner product dimensions and effective rank constraints. - **DoRA Magnitude Vectors:** For Weight-Decomposed LoRA (DoRA), extract directional low-rank tensors and companion magnitude scaling vectors (`lora_magnitude_vector`). ### 3. Base Model Binding & Compatibility Verification - **Layer Stride Verification:** Match adapter module paths (e.g. `base_model.model.model.layers.31.self_attn.q_proj`) against target base model architectures (LLaMA, Mistral, Gemma, Qwen, DeepSeek). - **Dimension Compatibility:** Verify whether $d_{in}$ and $d_{out}$ match the base model's hidden dimension ($d_{model}$) and intermediate MLP dimensions. ### 4. Merge-Plan Reconstruction - **Mathematical Scaling Inversion:** Reconstruct the mathematical merge formula for each targeted layer: $$\Delta W = \left( \frac{\alpha}{r} \right) (B \cdot A)$$ - **Memory & Precision Footprint:** Calculate the memory and precision requirements necessary to fuse the adapter into base model weights without running PyTorch or Python. --- ## Acceptance Criteria 1. Introduce `analyze_lora_adapter` in REA CLI and MCP contracts, accepting adapter directories or files. 2. Accurately parse `adapter_config.json` across LoRA and DoRA synthetic test fixtures. 3. Pair `lora_A` and `lora_B` tensor arrays and verify inner rank dimension $r$. 4. Validate base model compatibility and report target layer coverage in structured evidence. 5. Compute the mathematical merge scaling factor and output the layer-by-layer merge plan. --- ## Related - #3323 [Feature] Inspect GGUF and safetensors model files without loading them - #3651 [Feature] LLM application reverse engineering - #3652 [Feature] PyTorch checkpoint reverse engineering

bsakaguchi referenced this issue

2026-10-17 21:10:51 +00:00

[PR #1376] [MERGED] feat(setup): add native Pi client support #2838

bsakaguchi referenced this issue

2026-10-17 21:11:58 +00:00

[PR #1511] [MERGED] test(clients): verify native Qwen Pi and Hermes workflows #2931

bsakaguchi referenced this issue

2026-11-01 21:12:30 +00:00

[GH-ISSUE #8408] [Design] Connect indirect call investigations to Mach-O slot bindings #6589