Metadata-Version: 2.5
Name: attribution-graph
Version: 0.6.0
Summary: Calibrated entity resolution for attribution investigations
Project-URL: Homepage, https://github.com/OWNER/attribution-graph
Project-URL: Documentation, https://github.com/OWNER/attribution-graph/blob/main/METHOD.md
Project-URL: Issues, https://github.com/OWNER/attribution-graph/issues
Author: Tushar Karumudi
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: attribution,entity-resolution,followthemoney,osint,record-linkage,threat-intelligence
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Information Technology
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Security
Requires-Python: >=3.11
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Description-Content-Type: text/markdown

# attribution-graph

[![CI](https://github.com/OWNER/attribution-graph/actions/workflows/ci.yml/badge.svg)](https://github.com/OWNER/attribution-graph/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/attribution-graph.svg)](https://pypi.org/project/attribution-graph/)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)

**Calibrated entity resolution for attribution investigations.**

Given observations about domains, accounts, seller IDs and registry records,
decide which of them denote the same real-world entity — with a probability you
can defend rather than an edge in a graph you can't.

No network I/O. Bring your own collectors.

```bash
pip install attribution-graph
```

## Why this exists

OSINT tooling is well served on *collection* — SpiderFoot, recon-ng, Amass and
theHarvester all acquire data competently — and on *storage*, where OpenCTI and
Maltego hold and visualize the result. The gap is in between: deciding what the
collected observations actually support.

Graph traversal is the usual answer and it fails on entity attribution for a
structural reason. Infrastructure identifiers are primary keys; entity
identifiers are evidence. Treating the second like the first produces three
predictable errors:

- **Unbounded transitivity.** A shares an IP with B, B shares a registrar with C,
  C shares an analytics ID with D. Every edge is true and the component grows
  until it's useless.
- **Correlated evidence counted as independent.** 400 commits from one repo look
  like 400 observations. They're one.
- **Uniform edge weight.** A shared GA4 measurement ID and a shared Cloudflare IP
  are the same edge in a graph database, and differ by six orders of magnitude in
  what they imply.

This library fixes all three. Full derivation in [METHOD.md](METHOD.md).

## The model

```
llr = reliability × weight × decay(age) × log(1 / selectivity)
```

**Selectivity is measured, not asserted.** It's the fraction of the corpus
carrying an identifier value, so a GA4 ID held by one site is worth ~13.8 nats
and a Cloudflare IP is worth ~0 — without anyone telling the system what
Cloudflare is.

**Correlated observations can't stack.** Every claim declares a
`correlation_group`; within a group, evidence aggregates as `max + log(1+n)`
rather than summing.

**Corroboration is structural.** The per-group cap (8.0 nats) is deliberately
smaller than the prior (−11.5 nats), so no single inferential source can carry a
merge on its own — enforced by arithmetic, not analyst discipline.

**Registry assertions are exempt.** GLEIF saying an LEI belongs to a legal name
is definitional, not inferential. Without that carve-out the system could never
resolve a company to its own LEI.

## Quickstart

```python
import asyncio
from attribution_graph import (
    CaseScope, Engine, Claim, Identifier, IdKind, Predicate, Reliability,
    SourceClass, assess, write_all,
)

class MyCollector:
    name = "my_registry"
    accepts = (IdKind.ORG_NAME,)
    source_class = SourceClass.PUBLIC_REGISTRY   # checked at load; denied classes raise
    priority = 2

    async def collect(self, ident):
        return [Claim(
            subject=ident,
            predicate=Predicate.SAME_AS,
            object=Identifier(IdKind.LEI, "5493001KJTIIGC8Y1R12"),
            collector=self.name,
            source_url="https://api.gleif.org/...",
            reliability=Reliability.AUTHORITATIVE,
            correlation_group="gleif|5493001KJTIIGC8Y1R12",   # get this right
        )]

scope = CaseScope.load("case.yaml")     # refuses to load without an authorization ref
engine = Engine(scope, collectors=[MyCollector()])
result = asyncio.run(engine.run())
write_all(engine.graph, result, scope, Path("./out"))
```

Or score a set of claims directly, with no engine:

```python
from attribution_graph import assess

a = assess(claims, holder_lookup=lambda ident: corpus.count(ident))
print(a.probability, a.band.value, a.estimative, a.independent_groups)
# 0.989 ATTRIBUTED 'almost certainly' 2
```

## What it produces

- `investigation_graph.json` — claims with full provenance
- `entities.ftm.json` — [FollowTheMoney](https://followthemoney.tech/), loads into
  yente and Aleph
- `graph.cypher` — Neo4j
- `attribution_report.md` — ICD 203 estimative language, blocked merges, evidence
  groups

## The one thing to get right

`correlation_group`. It defines what counts as one observation, and it's the
difference between a calibrated score and a confident wrong answer. All SANs on
one certificate are one group. All profiles found by enumerating one username are
one group — the observation is *"this person reuses a handle"*, made once.

## Selectivity needs a corpus

`InMemoryIndex` (the default) counts only what the current case has seen, so it
systematically overestimates uniqueness. Assessments produced with it are **upper
bounds on confidence.** Implement `SelectivityIndex` against a persistent
observation store before treating output as evidential:

```python
class CorpusIndex:
    def holders(self, ident) -> int: ...
    def universe(self) -> int: ...

engine = Engine(scope, collectors, index=CompositeIndex(CorpusIndex(), InMemoryIndex(graph)))
```

## Scope is executable, not documentary

Attribution machinery generalizes across targets by construction: what identifies
a scraper operator identifies anyone. So the constraints are code, not a runbook.

- `CaseScope.load()` refuses to start without an authorization reference
- The engine raises on any collector declaring a denied source class — data
  brokers, breach corpora, authenticated scraping, biometrics, location brokers
- `pivot_radius` hard-caps hops from an authorized seed
- A case scoped to `Company` won't instantiate `Person` entities
- Identifiers are salted-hashed at rest under `minimize: true`
- Every collector call hits an append-only audit log

The deny list lives in the loader rather than the documentation because a
pipeline that *can* reach a source eventually will.

## Verification trail

What a reviewing investigator needs to check a report *now*: the reasoning in the
order it happened, an exact timestamp on every item, and numbered citations tying
each assertion to its source.

```python
trail = Trail(case_ref, authorization)
trail.seed("domain:scraper-site.example")
n = trail.cite("https://scraper-site.example/ads.txt", body_sha256=h, status=200)
trail.extract("seller ID", "pubmatic.com/156423", n)
trail.empty("https://api.ch/search?q=Acme", "UK Companies House", "companies_house_uk")
trail.infer("domains share an operator", basis="shared AdSense account",
            citations=[n])
```

Renders as an ordered, per-step-timestamped list with a numbered source appendix.
Steps are typed, so *(inference)* lines are visibly distinct from observations —
a reviewer can see at a glance which lines are things seen and which are
conclusions drawn. Queries that returned nothing are recorded as steps, because
a reviewer must be able to tell "checked and empty" from "never checked".

## Name variants across scripts

When two names match through transliteration, the report states **the exact
transform chain**:

```python
>>> name_match("Lakshmi", "Laxmi").describe()
"matched at 'lakshmi': 'Laxmi' via indic_ksh (1 transform)"
>>> name_match("Мосэнерго", "Mosenergo").describe()
"matched at 'mosenergo': 'Мосэнерго' via cyrillic_bgn_pcgn (1 transform)"
>>> name_match("Zhang Wei", "Chang Wei").describe()
"matched at 'chang wei': 'Zhang Wei' via pinyin_wade_giles (1 transform)"
```

23 named rules across Latin, Indic, Cyrillic (BGN/PCGN, ISO 9, ALA-LC), Arabic
and CJK (Pinyin/Wade-Giles, Hepburn/Kunrei, RR/McCune-Reischauer). Rule-based
rather than learned, deliberately: a model that cannot say *why* two names
matched is unusable in a report.

Over-generation is the real risk, so matches that collapse both names to a stub
are rejected — without that guard two shortening rules meet in the middle and
report a match between unrelated people.

## Negative evidence

Three things get conflated under "found nothing", and they carry different
weight: **not looked for** (no information), **looked for and absent** (weak
evidence against, scaled by source completeness), and **expected and absent** (the
hypothesis predicted it — the strongest negative signal available, and the one
nothing else models).

Coverage is declared per source rather than assumed. A UK company absent from
Companies House is meaningful; absent from Wayback means almost nothing.

## Evidence packages

For work that may reach counsel, `EvidenceLog` preserves what a report can only
describe:

```python
log = EvidenceLog(scope, Path("./out/evidence"))
log.record(url, status, body, collector="gleif")
log.record_negative(url, "companies_house_uk", "no matching entity on 2026-08-17")
write_evidence_package(log, findings_summary)
```

Produces content-addressed capture storage, a hash chain (removing or reordering
any capture breaks every subsequent entry), `evidence_manifest.json`, RFC 3161
timestamping instructions, a declaration skeleton, and a standalone
dependency-free `verify.py` a third party can run without installing anything.

**The framing that matters:** live web findings are not reproducible. Re-running
queries next year will not return this year's answers, and any report implying
otherwise is misleading. The preserved bodies are the evidentiary artifact; the
steps explain the reasoning. The manifest says this in as many words.

It also keeps observation separate from opinion. Captures are facts about what a
source contained at a time. Confidence scores are analysis — and per the
calibration gap below, analysis with no measured error rate. Don't let a report
blur the two.

## Companion packages

- **[adtx-attribution](https://github.com/OWNER/adtx-attribution)** — collectors
  for ads.txt/sellers.json, GLEIF, EDGAR, Companies House, RDAP, CT logs, plus a
  reverse seller-ID index that doubles as a selectivity corpus.
- **[handle-correlation](https://github.com/OWNER/handle-correlation)** —
  same-actor scoring for usernames observed across forums, code hosts and
  messaging platforms.

## Reports

Confidence scores are optional (`show_scores=False`) — sometimes the finding and
its sources are the deliverable and a probability just invites false precision.
When scores are shown, every report carries a disclaimer stating they are
analytical judgment rather than measured fact, and that the model is not yet
calibrated against ground truth. Every report ends with a per-source attribution
and terms table covering only the sources actually used.

## Status

`0.2.0`. The API is not yet stable. Calibration has not been validated against a
labelled ground-truth corpus — bands are principled but not empirically fitted,
so read probabilities as ordinal for now. That validation is the top open item;
see [METHOD.md §7](METHOD.md) and issue #1.

## License

Apache-2.0. See [CITATION.cff](CITATION.cff) if you use this in published work.
