Metadata-Version: 2.4
Name: modeldna
Version: 0.1.0
Summary: Forensic lineage detection for open-weight LLMs: fingerprint a model from its weights and find out what it really descends from.
Author-email: Muhammad Awais Bin Adil <binadilawais@gmail.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/AwaisAdilKhokhar/modelDNA
Project-URL: Repository, https://github.com/AwaisAdilKhokhar/modelDNA
Project-URL: Issues, https://github.com/AwaisAdilKhokhar/modelDNA/issues
Project-URL: Paper, https://arxiv.org/abs/2607.10617
Project-URL: Demo, https://huggingface.co/spaces/AwaisAdilKhokhar/modelDNA
Project-URL: Dataset, https://huggingface.co/datasets/AwaisAdilKhokhar/modeldna-atlas
Keywords: llm,fingerprinting,provenance,lineage,safetensors,gguf,forensics
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: requests>=2.28
Requires-Dist: huggingface_hub>=0.20
Requires-Dist: typer>=0.9
Requires-Dist: rich>=13.0
Requires-Dist: gguf>=0.10
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Dynamic: license-file

# modelDNA

[![arXiv](https://img.shields.io/badge/arXiv-2607.10617-b31b1b.svg)](https://arxiv.org/abs/2607.10617)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21305586.svg)](https://doi.org/10.5281/zenodo.21305586)

[![The Atlas: an interactive lineage constellation of 55 Hub models](https://raw.githubusercontent.com/AwaisAdilKhokhar/modelDNA/main/.github/assets/atlas.gif?v=2)](https://awaisadilkhokhar.github.io/modelDNA/)

**A 23andMe for open-weight models.** modelDNA fingerprints an LLM from its
weights alone and tells you, with a calibrated probability, which base model
it descends from — regardless of what the README claims.

```
$ modeldna scan some-org/suspicious-model

  SAME_LINEAGE  99.3% likely derived from Qwen/Qwen2.5-14B

  claimed lineage  "from scratch"   [!] INCONSISTENT
  README claims from-scratch training but weights are statistically
  consistent with derivation from Qwen/Qwen2.5-14B (p=0.993)

  signal                                   value   unrelated background
  attention sigma-curve correlation (F1)   0.994              0.30-0.70
  norm/bias vector cosine (F2)             0.995              0.85-0.92
  sampled parameter cosine / PCS (F3)      0.694             -0.01-0.01
  SVD spectrum correlation (F4)            0.897              0.95-0.99
```

Hugging Face's model lineage graph is self-reported: the `base_model`
metadata field is optional and unverified, over 60% of Hub models have no
documented parentage at all, and every laundering controversy so far
(Pangu/Qwen 2025, Reflection-70B 2024, Llama3-V 2024) was investigated ad hoc,
by hand. The detection methods are published and validated — modelDNA
packages them into one command.

**[Explore the Atlas](https://awaisadilkhokhar.github.io/modelDNA/)** — an
interactive family tree of 55 real Hub models reconstructed from weight
evidence alone: hover any edge for the σ-curve, sample-cosine, and spectral
evidence behind it — including the two decomposed merges, drawn as purple
edges labeled with their fitted mixture weights. Regenerate it yourself with `python scripts/build_atlas.py`.

**[Scan a model live](https://huggingface.co/spaces/AwaisAdilKhokhar/modelDNA)** —
paste a Hub repo id into the hosted scanner and get a verdict in a minute or
two, no install needed.

**[Read the tech report](https://github.com/AwaisAdilKhokhar/modelDNA/releases/download/report-v1.0/modeldna-tech-report.pdf)**
([arXiv:2607.10617](https://arxiv.org/abs/2607.10617)) —
*modelDNA: Calibrated Lineage Verification and Merge Decomposition from
Sampled Weight Fingerprints* (v1.0, July 2026): the fingerprint design, the
calibrated verdict engine, the merge-decomposition math, and every benchmark
number, with sources in [`paper/`](paper/).

**[Build on the data](https://huggingface.co/datasets/AwaisAdilKhokhar/modeldna-atlas)** —
the fingerprint DB and inferred lineage graph behind the Atlas, published as
a Hugging Face dataset (`models` / `edges` / `pairs` configs plus the raw
fingerprints and the pullable reference DB), refreshed weekly by CI.

## Install

```
pip install modeldna
```

Python 3.10+. CPU only — no GPU or torch required.

## Quick start

```bash
# pull the prebuilt reference DB of known bases (~57 MB, seconds)
modeldna db pull

# …or fingerprint the seed list from the Hub yourself
modeldna db build --limit 10   # quick smoke test, a few minutes
modeldna db build              # full seed list — resumable if interrupted

# scan a model against the DB
modeldna scan mistralai/Mistral-7B-Instruct-v0.3

# test a specific hypothesis pair (the Pangu/Qwen workflow)
modeldna compare suspect-org/model Qwen/Qwen2.5-14B

# what's actually in this merge? estimate the parent mixture
modeldna decompose org/mystery-merge org/hermes-ft org/dolphin-ft \
    --base mistralai/Mistral-7B-v0.1 --layers --mergekit

# machine-readable output, HTML evidence report
modeldna scan org/model --json
modeldna scan org/model --report report.html

# export a fingerprint without judging it
modeldna fingerprint org/model -o fp.json

# raw numbers behind the last verdict
modeldna explain org/model
```

Local model directories work anywhere a repo id does.

## How it works

The safetensors format stores a JSON header (tensor names, shapes, byte
offsets) at the start of each shard, and the Hub serves byte ranges over
HTTP. modelDNA exploits this to fingerprint a 7B model from roughly
100–300 MB of traffic instead of 15 GB:

- **Tier 0 — structure (kilobytes).** `config.json`, tokenizer hash, and the
  exact tensor inventory from shard headers. Filters the reference DB down to
  shape-compatible candidates and catches renamed mirrors outright.
- **Tier 1 — weight statistics (fast mode, ~100–300 MB).** All 1-D tensors
  (norm gains, attention biases) in full, plus seeded deterministic slices of
  every attention/MLP projection.

Four signal families are extracted and compared, following the published
literature:

| signal | method | after | survives |
|---|---|---|---|
| F1 | per-layer σ-curves of Q/K/V/O projections | arXiv:2507.03014 | SFT, continued pretraining, LoRA-merge |
| F2 | norm-gain / bias vector cosines | arXiv:2507.03014 | heavy training |
| F3 | seeded sampled-parameter cosine (PCS) | HuRef, arXiv:2312.04828 | SFT; the cheapest decisive test |
| F4 | top-k singular-value spectra | arXiv:2511.06390 | permutation / rotation re-parameterization |

Sample positions are pure functions of (seed, canonical role, layer, tensor
size), so any two models with matching shapes are sampled at identical
element positions — the reference DB stores only the samples, never full
weights, which keeps it in the single-digit MB per model.

GGUF repos (the llama.cpp / ollama ecosystem) work the same way: quantized
tensors are block-formatted, so an element range maps to a byte range of
whole blocks — modelDNA fetches the covering blocks, dequantizes with
llama.cpp's reference kernels, and samples the very same positions. A
Q4_K_M download compares straight against the fp16 original in the DB;
when a repo ships several quants, the highest-fidelity file is used.

A logistic model over the evidence vector produces the calibrated
probability, and every reported number ships with the unrelated-pair
background distribution measured on the DB itself, because a bare "0.93"
convinces nobody.

### Merge decomposition

Merges are the hard case for lineage — and the question the model-merging
community actually asks is "what's *in* this merge?". `modeldna decompose`
answers it from fingerprints alone:

```
$ modeldna decompose org/mystery-merge org/hermes-ft org/dolphin-ft --base mistralai/Mistral-7B-v0.1

  MERGE  weights are consistent with a multi-parent merge of the given parents

  candidate                    weight   ± roles   F2 check
  org/hermes-ft                +0.650     0.003      0.646
  org/dolphin-ft               +0.350     0.003      0.353
  mistralai/Mistral-7B (base)  +0.000     0.001     -0.001

  reconstruction cosine 0.99999   best single candidate org/hermes-ft at 0.99977
  mixture removes 94.0% of the residual the best single candidate leaves
```

The math: mergekit's mainstream methods — linear, slerp, task arithmetic,
TIES/DARE — all produce per-tensor weights that are (near-)linear
combinations of the parents, and PCS sample positions are pure functions of
(seed, role, layer, tensor size), so the same identity holds on the sampled
elements. Mixture weights fall out of a least-squares fit on data the
fingerprints already contain. Because merge parents are usually fine-tunes
of one base (raw cosine 0.99+, hopelessly collinear), the fit is constrained
to sum to one — algebraically a regression in *task-vector space*, where
fine-tune directions are nearly orthogonal and the system is
well-conditioned.

**Validated against real published mergekit configs**
(`benchmarks/merge_decompose_bench.py`, fingerprints cached in the repo):

- `mlabonne/NeuralPipe-7B-slerp` — the per-layer fit recovers the model
  card's *opposite* attention/MLP t-curves (anchors `[0, .5, .3, .7, 1]` and
  the reverse) at **r = 0.999** (attention) and 0.97 (MLP); the only
  deviations sit at layers where the parents are near-identical, i.e. where
  the weight is mathematically unidentifiable.
- `mlabonne/Monarch-7B` — a dare_ties merge; fitted weights
  **0.371 / 0.347 / 0.291** vs the published `0.36 / 0.34 / 0.30`
  (max error 0.011), base column correctly ~0. The summary hedges to
  AMBIGUOUS because the Beagle parents themselves share ancestry (task
  vectors r = 0.987 — the collinearity warning fires), and DARE
  sparsification bounds the residual shrink: the weights are right, and the
  hedge is the tool being honest about what the weights alone can prove.
- Chain closure: `mlabonne/AlphaMonarch-7B` puts **α = 1.000** on its true
  ancestor Monarch-7B once Monarch is in the candidate list, and 0 on the
  decoys.

What you get beyond the weights: a per-layer mixture profile (`--layers`;
gradient/slerp merges show as depth-varying α), an optional base column
(`--base`) that absorbs un-finetuned mass so task-arithmetic and low-density
DARE merges decompose cleanly, an independent cross-check fitted on the 1-D
norm vectors (F2), a residual statistic that separates "exact merge" from
"merge + fine-tune", and `--mergekit` to print the nearest linear mergekit
config for the fitted mixture. Parents already in the reference DB are not
re-downloaded.

### Verdict classes

Every scan resolves to exactly one class:

| class | meaning |
|---|---|
| `EXACT_COPY` | weights byte/near-identical to a DB entry (re-upload or renamed mirror) |
| `QUANTIZED_COPY` | matches a DB entry within quantization noise |
| `FINE_TUNE` | small parameter delta from one parent (SFT/DPO/LoRA-merged) |
| `SAME_LINEAGE` | large delta but lineage signals match one parent (continued-pretrain regime) |
| `LIKELY_MERGE` | evidence splits across ≥2 parents |
| `SAME_FAMILY_UNRESOLVED` | family-level match, exact parent ambiguous |
| `NO_MATCH` | no DB parent supported; consistent with independent training or an unindexed base |
| `INSUFFICIENT` | unsupported format/architecture, or data could not be read |

The tool prefers honest abstention over confident error: thresholds are
conservative, and the `README`-vs-weights consistency flag fires only when
the repo *makes a claim* that the weight evidence contradicts at high
confidence.

### Exit codes (CI-friendly)

- `0` — weight evidence consistent with the repo's claim (or no claim made)
- `3` — weight evidence contradicts the claimed lineage
- `4` — lineage unknown (no DB match, or unsupported format)

## Reference DB

The fastest way to a working DB is to pull a prebuilt one — fingerprints
are small (~55 MB for the full seed list) and derived from public weights:

```bash
modeldna db pull                 # seconds; then a scan only fetches your suspect
```

`db pull` fetches the pinned reference-DB release archive from this repo's
GitHub releases (override with `--url` or `MODELDNA_DB_URL`; archives are
produced by `modeldna db export` / the `refdb-release` workflow). To build it
yourself instead:

`modeldna db build` fingerprints a seed list of widely-forked bases (Llama,
Qwen, Mistral, Gemma, Phi, DeepSeek, Yi, Falcon, OLMo, …) plus deliberate
hard negatives — OpenLLaMA (same shapes as Llama, independently trained) and
Pythia seed siblings. Gated repos (Llama, Gemma) need a Hugging Face token
with the license accepted on each gated org (`huggingface-cli login` or
`HF_TOKEN`) — a read-only token is sufficient, no write scope needed.

The seed list has been run against the live Hub: **37 of 41 entries
indexed**, ~55 MB total. The 4 that don't index
(`deepseek-ai/deepseek-llm-7b-base`, `internlm/internlm2-7b`,
`openbmb/MiniCPM-2B-sft-bf16`, `openlm-research/open_llama_7b`) ship only
PyTorch `.bin` checkpoints on the Hub, no safetensors — a hard limitation of
the current safetensors-only I/O layer, not a transient failure. Fast mode
plans every byte range it needs up front and fetches them concurrently
(measured: a 7B model in ~2 min on an ordinary connection), so a full build
is a coffee break, not an overnight job. If your connection drops mid-run,
just rerun — `db build` skips already-indexed entries automatically. Two env
knobs if you need them: `MODELDNA_HTTP_CONCURRENCY` (parallel range reads,
default 32) and `MODELDNA_COALESCE_GAP` (merge nearby ranges into one
request, default 64 KiB — raise it on high-latency, high-bandwidth links).

```bash
modeldna db list                 # what's indexed
modeldna db info Qwen/Qwen2.5-7B
modeldna db add org/new-base --family my-family
modeldna db remove org/old-base
modeldna db export refdb.tar.gz  # package for publishing / air-gapped machines
modeldna db pull --url refdb.tar.gz --overwrite
```

There is also a manual `fingerprint` GitHub Actions workflow: give it Hub
repo ids and it extracts fingerprints on a runner sitting next to the HF
CDN (seconds per model) and uploads them as artifacts — useful for growing
the DB or the benchmark without fetching through your own connection.

The DB lives in `~/.modeldna/db` (override with `MODELDNA_DB`). Entries are
gzipped JSON fingerprints plus a manifest — no weights are ever stored.

## Benchmark

`benchmarks/synthetic_bench.py` runs the evaluation harness on generated
models (independent bases vs fine-tunes, heavy continuations, and fp16
re-casts) and enforces the ship gates in CI: AUROC ≥ 0.99, TPR ≥ 0.95 at
FPR ≤ 1%, and **zero false positives on independent same-architecture
controls at the reporting threshold** — that last gate is the
defamation-risk surface and is non-negotiable. Current results: AUROC 1.0,
0 false positives, max calibration gap 0.045.

**Real-model LineageBench** (`benchmarks/real_lineagebench.py`) runs the
same gates on real Hub weights: 15 suspect models across 6 families whose
parentage is corroborated by the publishing org's own documentation
(fine-tunes, continued-pretrains, official instructs, a merge chain, a
depth-upscale, a GPTQ quantized copy), each judged against all 8 candidate
parents by the actual verdict engine, with 107 wrong-parent and
cross-family comparisons as hard negatives. Results (2026-07-07):

- **AUROC 1.0 · TPR@FPR1% 1.0 · 0 false positives at the 0.9 threshold** —
  ship gates PASS. Weakest positive scored 0.9426 (gemma-2b-it); strongest
  negative 0.6596: clean separation.
- **Top-1 parent attribution: 13/13** on the gate pool — every derivative
  traced to the right base among 8 candidates.
- The two by-design limitation cases behaved exactly as documented: SOLAR
  (depth-upscale) → honest `NO_MATCH` abstention; GPTQ (quantized-copy) →
  abstained on packed int4 tensors while still ranking the true parent
  first.
- One caveat, reported not hidden: the merge-chain case (AlphaMonarch)
  resolved to the correct family as `SAME_FAMILY_UNRESOLVED` rather than
  `LIKELY_MERGE` — consistent with the stated v0.1 limitation that merges
  are flagged, not attributed.

The ground truth manifest (`benchmarks/lineagebench_pairs.json`), the
fingerprint cache, and full per-pair results
(`benchmarks/lineagebench_results.json`) are committed, so
`python benchmarks/real_lineagebench.py --no-fetch` reproduces the numbers
without touching the Hub.

LineageBench is published as a named public dataset —
[**AwaisAdilKhokhar/lineagebench**](https://huggingface.co/datasets/AwaisAdilKhokhar/lineagebench)
(ground truth with per-pair citations + this tool's reference results) — and
written up in [`paper/lineagebench.md`](paper/lineagebench.md). Rebuild and
republish the dataset with `python scripts/export_lineagebench.py --push`.

## Limitations — read before citing a verdict

- **Statistical evidence, not proof.** Output is always "weights are
  statistically consistent with derivation from X", never an accusation.
- **Distillation is invisible** to weight forensics by construction; every
  weight-space method fails on it. Behavioral methods are on the roadmap and
  will be labeled experimental.
- **Merge attribution needs a candidate list.** `modeldna decompose`
  estimates mixture weights for parents you name; a scan alone flags
  `LIKELY_MERGE` but cannot enumerate parents the DB has never seen.
  TIES/DARE sparsification and post-merge fine-tuning show up as residual,
  not as exact weights; depth-changing merges (passthrough / frankenmerge)
  are rejected, not decomposed.
- **Adversarial robustness is bounded.** Deliberate permutation/rotation
  re-parameterization defeats direct-similarity signals (F1–F3); spectral
  invariants (F4) are the counter, and the report surfaces "low direct
  similarity, high invariant similarity" as an anomaly pattern that is
  itself informative. A determined, retraining-scale adversary is out of
  scope and stated so in every report.
- Depth-upscaled / layer-surgery derivatives are detected as shape mismatches
  and reported, not resolved (alignment search planned for v0.3).
- **Safetensors and GGUF.** GGUF repos are fingerprinted by dequantizing
  the sampled positions (any ggml quant type, K-quants and IQ included;
  llama.cpp's q/k row permutation is undone so positions align with the
  original weights). GPTQ/AWQ packed-int4 safetensors still abstain, and
  repos that ship exclusively PyTorch `.bin` / `.pt` checkpoints
  (`deepseek-llm-7b-base`, `internlm2-7b`, `open_llama_7b`,
  `MiniCPM-2B-sft` among the seed list) can't be fingerprinted at all
  until `.bin` decoding is added.

## Library use

```python
from modeldna.scan import scan, compare_pair
from modeldna.db.store import ReferenceDB

res = scan("org/model", db=ReferenceDB())
print(res.verdict.verdict, res.verdict.probability)
print(res.to_dict())

fp_a, fp_b, evidence, verdict = compare_pair("org/suspect", "org/alleged-parent")
print(evidence.sigma_r, evidence.pcs_cos_mean)

from modeldna.merge import decompose_targets

dec = decompose_targets("org/mystery-merge", ["org/parent-a", "org/parent-b"],
                        base="org/shared-base", db=ReferenceDB())
print(dec.summary, dec.alphas, dec.gain_vs_single)
```

## Development

```bash
git clone https://github.com/AwaisAdilKhokhar/modelDNA
cd modelDNA
pip install -e .[dev]
pytest -q
ruff check src tests benchmarks
python benchmarks/synthetic_bench.py
```

## References

The methods productized here come from, among others:

- HonestAGI, *Intrinsic Fingerprint of LLMs* (arXiv:2507.03014) — attention σ-curves
- Zeng et al., *HuRef* (NeurIPS 2024, arXiv:2312.04828) — PCS/ICS comparators
- *Ghost in the Transformer* (arXiv:2511.06390) — invariant spectral signatures
- Horwitz et al., *Model Tree Heritage Recovery* (arXiv:2405.18432) and
  *Model Atlas* (arXiv:2503.10633) — population-scale lineage recovery
- *Are Robust LLM Fingerprints Adversarially Robust?* (arXiv:2509.26598) — the threat model

See the [technical report](https://github.com/AwaisAdilKhokhar/modelDNA/releases/download/report-v1.0/modeldna-tech-report.pdf)
(Section 2) for the full research review.

## Citing

If you use modelDNA, the atlas dataset, or LineageBench, please cite the
technical report
([arXiv:2607.10617](https://arxiv.org/abs/2607.10617)
· [PDF](https://github.com/AwaisAdilKhokhar/modelDNA/releases/download/report-v1.0/modeldna-tech-report.pdf)
· [DOI 10.5281/zenodo.21305586](https://doi.org/10.5281/zenodo.21305586)):

```bibtex
@techreport{modeldna2026,
  title         = {modelDNA: Calibrated Lineage Verification and Merge
                   Decomposition from Sampled Weight Fingerprints},
  author        = {Muhammad Awais Bin Adil and Saad Aamir},
  year          = {2026},
  month         = {July},
  eprint        = {2607.10617},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  doi           = {10.5281/zenodo.21305586},
  url           = {https://arxiv.org/abs/2607.10617},
  note          = {v1.0. Live scanner: huggingface.co/spaces/AwaisAdilKhokhar/modelDNA}
}
```

## License

Apache-2.0
