Metadata-Version: 2.5
Name: mergelens
Version: 2.0.0
Summary: Experimental static inspection toolkit for homologous LLM checkpoints.
Project-URL: Homepage, https://github.com/shuhulx/mergelens
Project-URL: Repository, https://github.com/shuhulx/mergelens
Project-URL: Issues, https://github.com/shuhulx/mergelens/issues
Author-email: Shuhul Razdan <shuhul.aiml@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: compatibility,deep-learning,diagnostics,llm,machine-learning,mergekit,model-merge,model-merging,neural-network,transformers,weight-analysis
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.20
Requires-Dist: numpy>=1.24
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: safetensors>=0.4
Requires-Dist: torch>=2.0
Requires-Dist: typer>=0.9
Provides-Extra: all
Requires-Dist: jinja2>=3.1; extra == 'all'
Requires-Dist: mcp<2,>=1.12.4; extra == 'all'
Requires-Dist: plotly>=5.18; extra == 'all'
Provides-Extra: dev
Requires-Dist: build; extra == 'dev'
Requires-Dist: mypy<2,>=1.19; extra == 'dev'
Requires-Dist: numpy<2.5; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff<0.17,>=0.16.4; extra == 'dev'
Requires-Dist: types-pyyaml; extra == 'dev'
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.12.4; extra == 'mcp'
Provides-Extra: report
Requires-Dist: jinja2>=3.1; extra == 'report'
Requires-Dist: plotly>=5.18; extra == 'report'
Description-Content-Type: text/markdown

# MergeLens

[![CI](https://github.com/shuhulx/MergeLens/actions/workflows/ci.yml/badge.svg)](https://github.com/shuhulx/MergeLens/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/mergelens)](https://pypi.org/project/mergelens/)
[![Python](https://img.shields.io/pypi/pyversions/mergelens)](https://pypi.org/project/mergelens/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue)](https://github.com/shuhulx/mergelens/blob/main/LICENSE)

MergeLens is an experimental inspection toolkit for homologous LLM checkpoints. It reports exact tensor coverage and weight, spectral, task-vector, and optional activation-similarity signals; highlights tensors worth inspecting; and proposes a rule-based MergeKit starting configuration. It is intended for model-merging researchers and engineers who need inspectable evidence before spending compute on candidate merges.

MergeLens does **not** establish downstream merged-model quality, capability retention, or the best merge method. Its aggregate score and thresholds are hand-specified, unvalidated heuristics. Post-merge behavioural evaluation remains necessary.

Run a local software demonstration with tiny synthetic safetensors:

```bash
pip install -e .
pip install -e '.[report]'
python examples/synthetic_demo.py --output-dir demo-output
```

The example is a software and known-answer demonstration, not scientific validation of merge outcomes.

## Install

```bash
pip install mergelens
pip install 'mergelens[report]'  # offline HTML reports
pip install 'mergelens[mcp]'     # MCP server
```

## Compare checkpoints

```bash
mergelens compare model_a/ model_b/
mergelens compare finetune_a/ finetune_b/ --base shared_base/
mergelens compare model_a/ model_b/ --metric cosine_similarity --metric l2_distance --json result.json
mergelens compare model_a/ model_b/ --report report.html
```

The result exposes:

- the reference and candidate identity for every pair-tensor row;
- separately attributable `candidate_set_metrics` for sign/TSV and `activation_metrics` for CKA;
- total tensors and parameters, missing names, shape/dtype issues, and exact comparable coverage;
- architecture metadata and known structural incompatibilities;
- a status and reason for every computed, skipped, unavailable, failed, or resource-limited signal;
- raw parameter-weighted heuristic components, with partial availability reducing component weight by parameter coverage;
- a machine-readable `validation_status: heuristic_unvalidated`;
- pair-bounded tensor inspection regions; and
- an illustrative or parser-validated MergeKit starting configuration.

Structurally unsupported comparisons retain their raw coverage and measurements but suppress aggregate scoring.

Python API:

```python
from mergelens import compare_models

result = compare_models(["model_a/", "model_b/"])

print(result.coverage[0].parameter_coverage_reference)
print(result.mci.score)  # float or None when suppressed
print(result.mci.risk_tier)
print(result.mci.validation_status)  # heuristic_unvalidated

for row in result.tensor_metrics:
    print(row.reference_model, row.candidate_model, row.tensor_name, row.cosine_similarity)

for row in result.candidate_set_metrics:
    print(row.base_model, row.candidate_models, row.tensor_name, row.sign_disagreement_rate)

for row in result.activation_metrics:
    print(row.comparison_id, row.activation_layer, row.cka_similarity, row.warnings)

for signal in result.metric_availability:
    print(signal.metric, signal.status.value, signal.reason)
```

## Nine underlying diagnostic signals

The composite heuristic is not counted as a separate diagnostic signal.

| Signal | Level | Direct object | Default | Composite |
|---|---|---|---|---|
| Cosine similarity | Weight | Exact-shape flattened tensor alignment | Yes | Yes |
| Normalized L2 distance | Weight | Difference relative to average tensor norm | Yes | No; displayed raw |
| Weight-distribution divergence | Weight, experimental | Directional softmax transform of flattened weights | No; explicit selection only | No |
| Spectral overlap | Weight | Leading left-singular-subspace overlap for matrices | Yes, resource bounded | Yes |
| Effective-rank ratio | Weight | Ratio of entropy-derived effective ranks | Yes, resource bounded | Yes |
| Sign disagreement | Candidate set task vector | Pairwise sign mismatch; zero/nonzero counts as mismatch | Yes when a shared base and at least two candidates exist | No |
| TSV interference | Candidate set task vector | Pairwise numerical-rank right-subspace overlap | Yes when a shared base and at least two candidates exist | No |
| Task-vector energy | Pair tensor task vector | Fraction of spectral energy in retained leading values | Yes with an explicit base | No |
| Linear CKA | Activation layer, optional | Exact activation-layer observations with calibration and feature-width provenance | Only when supplied | No |

All SVD-backed signals use a conservative full-decomposition resource policy and retain only numerical-rank directions. Full-ambient subspaces are reported as uninformative. A metric skipped by the resource policy is reported as `resource_limit_skipped`; it is not silently converted into a plausible number.

## MergeKit configuration diagnosis

```bash
mergelens diagnose merge.yaml --json diagnosis.json
```

Diagnosis honours checkpoint references, an explicit task-vector base, and finite non-negative scalar weights only for top-level full-model inputs. It discloses ignored or unsupported semantics such as slice assembly, gradients, tokenizer remapping, chat templates, and method-specific merge execution. Unknown merge methods fail closed instead of becoming `linear`.

Generated configurations follow current MergeKit model/parameter placement. They are marked `schema_validated` only when the installed MergeKit parser accepted them; otherwise they are marked `illustrative`.

## Reports and MCP

`mergelens[report]` produces one HTML file with Plotly JavaScript embedded. Charts group by explicit comparison ID and preserve missing metric values.

The MCP server exposes seven tools: `compare_models`, `diagnose_merge`, `get_conflict_zones`, `suggest_strategy`, `generate_report`, `explain_layer`, and `get_compatibility_score`.

```json
{
  "mcpServers": {
    "mergelens": {
      "command": "mergelens",
      "args": ["serve"]
    }
  }
}
```

## Memory and reproducibility

Safetensors are memory-mapped and aligned tensor groups are consumed lazily. No exact peak-memory multiplier is claimed: runtime memory also includes float32 conversions, task vectors, bounded SVD workspaces, activation tensors, result rows, report data, and framework overhead.

See [limitations](https://github.com/shuhulx/mergelens/blob/main/LIMITATIONS.md), [validation status](https://github.com/shuhulx/mergelens/blob/main/VALIDATION.md), [migration guidance](https://github.com/shuhulx/mergelens/blob/main/MIGRATION.md), and the [changelog](https://github.com/shuhulx/mergelens/blob/main/CHANGELOG.md) before interpreting results.

## Development

```bash
python -m pip install -e '.[dev,all]'
ruff check .
ruff format --check .
pytest -q
mypy src/mergelens
python -m build
```

Supported Python versions are 3.10, 3.11, and 3.12.

## References

- MergeKit documentation and schema: [arcee-ai/mergekit](https://github.com/arcee-ai/mergekit)
- Kornblith et al., “Similarity of Neural Network Representations Revisited”: [arXiv:1905.00414](https://arxiv.org/abs/1905.00414)
- Yadav et al., “TIES-Merging”: [arXiv:2306.01708](https://arxiv.org/abs/2306.01708)
- Gargiulo et al., “Task Singular Vectors”: [arXiv:2412.00081](https://arxiv.org/abs/2412.00081)

## License

Apache-2.0. See the [license](https://github.com/shuhulx/mergelens/blob/main/LICENSE).
