# SOURCES — where the bundled data came from

Every table shipped in `src/tcren/data/` and loaded by the library. Each entry gives the origin, how
to re-fetch or regenerate it, and whether the values are **published** (transcribed from a paper or
database), **derived** (computed by us from structures), or **unresolved**.

The authoritative per-value detail lives in the loader docstrings in `src/tcren/potential/model.py`
— including the cross-checks. This file exists so the provenance question can be answered without
reading the code, and so an unrecorded origin is visible as such rather than assumed.

Structure datasets (`Native2022`, `Native2026`, `Canonical2026`, the modelled sets) are not here:
they are not bundled, they are fetched from Hugging Face, and the benchmark repository owns their dataset catalogue.

---

## Experimental contact-marginals resource — derived, not bundled

The opt-in `contact-prior-fit` CLI derives a separately versioned JSON resource
from exact external antibody atom contacts/current-coordinate frame verification
and fresh native `export --quantity contact-atoms --interface all` observations.
External deposit identity, license, download hashes and overlap policy are recorded
in `research archive: CONTACT_PRIOR_SOURCES.md`; model definitions and commands are
in `docs/contact-penalties.rst`. This does not regenerate either bundled TCRen table.

The resource retains actual external and native marginal counts/means/variance
sums, the pooled-chemistry backoff and external-support categorical Dirichlet
counts. The external counts supply the prior strength; this is an estimated
dependency. Normal transformed marginals, discarded torsion signs and correlated
within-complex observations limit interpretation. Posterior predictive tails are
compatibility scores, not calibrated naturalness/binding probabilities. All four
penalty constructions use the same frozen worst-supported geometry weight and
full-table potential maximum, fixed before benchmark accuracy inspection.

Exact regeneration requires the pinned raw deposit and current mmCIF source
manifests, complete native-ID exclusions, fresh native input hashes, producing
noneditable wheel and resource digest. Raw inputs and private benchmark mappings
remain in their owning storage; no cohort normalization or assay-fitted coefficient
is bundled. Reference and score outputs carry their hashes rather than version-only
claims. Missing matrix entries and recovered geometry are explicit.

## `sidechain_geometry.json` — derived

Side-chain internal coordinates: per residue type and atom, the bond length to its parent, the bond
angle, and the torsion offset. `tcren.refine.sidechain.build_sidechain` reads it to place every
heavy atom of any of the nineteen non-glycine types from a backbone and a chi vector.

- Origin: **derived by us** from the 374 `Canonical2026` crystals, over **176,475** residues. No
  external table is used and none is needed — the same arrangement as
  `tcren.geometry.BOND_CA` (3.7985 Å over 642 CDR gaps) and
  `tcren.refine.substitute.virtual_cb`, which is what keeps the builder consistent with the set the
  rest of the library is derived from.
- **Chemistry is declared, geometry is measured.** The connectivity — which atom hangs off which
  three, and which chi drives its torsion — is `sidechain.TOPOLOGY` in the source and carries no
  number. Only the numbers are fitted.
- Units: bonds in ångström, **angles and offsets in radians**. Circular quantities are circular
  means.
- Regenerate: `python scripts/fit_sidechain_geometry.py --structures data/Canonical2026 -o src/tcren/data/sidechain_geometry.json`
- Check: round-tripping a deposited residue through its own chi reads **0.0784 Å** mean heavy-atom
  RMSD (median 0.0466, p95 0.2323) over 54,954 residues; worst type ARG 0.2210 Å, best SER 0.0207 Å.
  `tests/unit/test_sidechain.py` asserts the round trip per type.
- Loader: `tcren.refine.sidechain.geometry()`.
- **Not from a rotamer library.** The chi grid `background.scan_structure` samples is a plain
  staggered grid (`-60 + k·step`, default 60°), not a backbone-dependent rotamer library, and it is
  calibrated against the invariant that a deposited residue be admissible at its own position
  (`scripts/calibrate_background_scan.py`). If a published rotamer library is ever adopted its
  citation belongs here, retrieved rather than written from memory.

## `TCRen_potential.csv` — derived

The classic TCRen TCR↔peptide potential. **Directed**: rows are the TCR side, columns the peptide
side, so it must not be symmetrised or decomposed.

- Bundled identity: the May 2026 updated classic table from commit
  `ef204be2f787ece0088cc2471c24079f135621a3`, copied into the Python package at
  `91f50249ae788e468e6735b6128472da347a9cbc`. SHA256:
  `1aba9dce3cf67600d1216f3dd533290f33f355c0df9cc282b084d85e8047794b`.
- Original-paper artifact: root and pipeline `TCRen_potential.csv` at December 2022
  commit `e2b6b95b6bdf876d6bd92f9eda733ec240cb43c3` have SHA256
  `d940e80abf7c9742c009fb463423a00758e3a277f77f750300c3ed24b3f3fea6`.
  All 380 values match the saved Figure 3 TCRen subset and the March 2026 root
  table. The contemporaneous pipeline copy differs; do not conflate variants.
- Reference metadata changes from 239 structures / 140 nonredundant representatives
  in the original snapshot to 333 / 196 in the May update. The original R recipe
  selects nonredundant representatives, counts directed contacts and adds one
  pseudocount before marginal normalization. Updated populations and alternative
  redundancy recipes can legitimately change coefficients; numerical differences
  alone do not establish a calculation error.
- Regeneration is not certified by a generic `derive-potential` call: exact input
  membership, representative selection and recipe/version must be pinned.
- Loader: `tcren.potential.tcren()` / CLI `karnaukhov2022` retains the bundled May
  table for compatibility. Its alias is not proof of original-paper identity.
  The original saved CSV can be supplied explicitly through `--potential CSV`;
  this provenance correction changes no coefficients or defaults.

## `TCRen2_potential.csv` — derived

The **default** TCR↔peptide potential since 2.11.0, and the matrix the TCRen2 manuscript reports.
**Directed**, like `TCRen_potential.csv`: rows are the TCR side, columns the peptide side.

- Origin: derived by us from the `Native2026` TCR:pMHC crystals, redundancy-balanced on both the
  epitope and the receptor axis. Since 3.3.0 the derivation runs on **359 of the 374** deposited
  structures — the unconditional αβ filter drops 12 (3 pMHC-only, 8 single-chain, 1 γδ) and the
  withdrawn-entry rule drops 3 more.
- Regenerate: `tcren fetch-data`, then
  `tcren derive-potential --structure-dir "$TCREN_DATA_DIR/Native2026" --balance both -o TCRen2_potential.csv`.
  `data/potentials.json` records the recipe verbatim and
  `tests/regression/test_shipped_potentials.py` reproduces the file from it.
- Loader: `tcren.potential.tcren2()`; it is the default the TCR:peptide interface scores under.

## `pdb_withdrawn.csv` — curated (2026-09-04)

The PDB entries the RCSB has withdrawn that appear in a bundled structure set: 4 rows, each with
its status, its successor if it has one, the reason, and the date the RCSB holdings endpoint was
checked. `derive-potential` drops them unless `--keep-withdrawn` is passed.

- Origin: verified one id at a time against the RCSB holdings endpoint
  (`https://data.rcsb.org/rest/v1/holdings/removed/<id>`), not taken on trust.
- Regenerate: re-query that endpoint for every id in the structure set; the file is hand-curated
  from its replies, so there is no single command.
- Loader: `tcren.potential.withdrawn_entries()` / `withdrawn_ids()`.

## `MJ_Keskin_potentials.csv` — published (identified 2026-08-29)

Loaded as both `mj()` and `keskin()`. **Every score in the package is built on this file.**

- Origin: carried over from the original R implementation with no record, and **identified against
  AAindex3 on 2026-08-29** by `tcren.potential.identify()`, which compares a matrix cell by cell
  against all 47 AAindex3 entries:
  - the `MJ` column is **`MIYS990106`** — Miyazawa S, Jernigan RL, *Proteins.* 1999;34(1):49–68,
    doi:10.1002/(SICI)1097-0134(19990101)34:1<49::AID-PROT5>3.0.CO;2-L (PMID 10336383). **400 of
    400 cells match exactly** (max |Δ| = 0); the next-closest AAindex3 entry is off by 0.65, and
    every other Miyazawa–Jernigan matrix in the file — `MIYS850102`, `MIYS850103`, `MIYS960101`,
    `MIYS960102`, `MIYS990107` — is off by 0.66 to 6.53. So it is **MJ 1999, not 1985 and not
    1996**, and the citation risk recorded here since 2026-08-11 is closed.
  - the `Keskin` column is **`KESO980101`**, the solvent-mediated form — 400 of 400 cells exactly,
    next-closest off by 2.77.
- Re-verify: `python -c "from tcren.potential import identify, mj, keskin; print(identify(mj())[:2], identify(keskin())[:2])"`
- What was established earlier and still holds: it is **not** MJ 1996 Table 3 — that table is
  attractive everywhere (Ala–Ala −2.72, range −7.37 to −0.12) while this one takes both signs
  (Ala–Ala −0.12, range −1.19 to +0.76). It correlates with Table 3 at r = 0.89 but is not its
  double-centred pair part either (r = 0.51).
- Status: left byte-for-byte untouched, since changing it would change every published number. It
  can now be cited as Miyazawa & Jernigan 1999.
- **The two potentials in this one file are in different reference states** (2026-08-29): the `MJ`
  side is mixed-sign with mean −0.079, the `Keskin` side is negative everywhere with mean −3.547.
  Swapping one for the other therefore changes the reference state as well as the derivation. The
  like-for-like partners are `MJ` ↔ `BT1999` and `Keskin` ↔ `MJ1996`.

## `MJ1996_contact_energies.csv` — published

Miyazawa–Jernigan 1996 inter-residue contact energies `e_ij` (RT units), Table 3.

- Origin: AAindex accession `MIYS960101`, transcribed from a published copy of Table 3 and
  cross-checked against a second independent copy — agreeing on the alphabet order
  `CMFILVWYAGTSNQDEHRKP`, on Ala–Ala (−2.72), and on the full range (−7.37 to −0.12).
- Re-fetch: <https://www.genome.jp/dbget-bin/www_bget?aaindex:MIYS960101>
- Excluded on purpose: the companion repulsive packing-density term of the same paper, which is a
  function of coordination number rather than of a residue pair.
- **Five cells disagree with AAindex's own `MIYS960101` (2026-08-29)**, by 0.04 to 0.28: M–V, D–M,
  E–M, H–R and A–P. 205 of the 210 unique pairs agree exactly and the two matrices correlate at
  0.99978. Four of the five involve Met, Arg or His, which reads like a transcription slip on our
  side rather than a different edition. **Left byte-for-byte untouched** under the same rule as the
  MJ/Keskin file, and pinned by `tests/unit/test_aaindex.py`. It feeds no manuscript number:
  `\PotPinGenericFoldMax` takes the max over three generic potentials and that max is `mj`, not
  this one. `aaindex("MIYS960101")` is the curated alternative when one is wanted.
- Reference: Miyazawa S, Jernigan RL. *J Mol Biol.* 1996;256(3):623–644. doi:10.1006/jmbi.1996.0114
- Loader: `tcren.potential.mj1996()`.

## `MJ1985_partition_energies.csv` — published

Effective partition energies, one per residue — the one-body term a pairwise matrix cannot supply.
Sign convention is opposite to a contact energy: larger is more hydrophobic.

- Origin: AAindex accession `MIYS850101`, retrieved from two endpoints of the AAindex database that
  returned identical values.
- Re-fetch: <https://www.genome.jp/dbget-bin/www_bget?aaindex:MIYS850101>
- Independent check: correlates at r = +0.98 with the hydrophobicity axis
  `Potential.hydrophobicity_fit()` recovers from `MJ1996_contact_energies.csv`, which was
  transcribed from a different source — a transcription error would break that agreement.
- Reference: Miyazawa S, Jernigan RL. *Macromolecules.* 1985;18:534–552.
- Loader: `tcren.potential.mj_partition_energy()`.

## `aaindex3.txt` — published (upstream flat file, verbatim)

The whole of AAindex3, GenomeNet's section of statistical protein contact potentials: 47 records
over the 20 amino acids, each transcribed by its curators from a published table. Bundled as the
**upstream file itself** rather than a converted subset, so provenance is the record and a reader
can diff it against a fresh download.

- Origin: <https://www.genome.jp/ftp/db/community/aaindex/aaindex3> (80 kB), fetched 2026-08-29.
- Re-fetch: `curl -sL https://www.genome.jp/ftp/db/community/aaindex/aaindex3 -o src/tcren/data/aaindex3.txt`
- Contents by kind, as `potential.catalogue()` reports them: **42 contact energies**, 2 contact-count
  tables (`TANS760102`, `MIYS960103`) and 3 side-chain-distance tables (`BONM030104`–`BONM030106`).
  `potential.aaindex()` refuses the last two kinds, because scoring a contact map with a count table
  is a silent category error. Three `ZHAC*` entries are asymmetric by construction (row secondary
  structure vs column secondary structure) and must not be decomposed.
- Caveat carried in the code: **AAindex's `R` (PMID) field sometimes cites the paper that tabulated
  a matrix rather than the one that derived it** — `MIYS850102` carries Bastolla 2001. Verify before
  citing; every entry's own `A`/`T`/`J` fields are the authors, title and journal to check against.
- Licence: AAindex is distributed publicly by anonymous FTP/HTTPS from GenomeNet and its contents are
  taken from published literature. **Redistribution terms are not stated on the download page**, so
  this bundling is flagged rather than settled.
- Loaders: `tcren.potential.aaindex(accession)`, `entry()`, `catalogue()`, `identify()`.

## `BT1999_contact_energies.csv` — published

Betancourt–Thirumalai contact energies, the `B` matrix (RT units): Miyazawa–Jernigan re-referenced
with **Thr as the reference solvent**, so every Thr entry is exactly `0.00`.

- Origin: AAindex3 accession `BETM990101` ("Modified version of the Miyazawa-Jernigan transfer
  energy"), parsed from the raw record rather than retyped — lower-triangular over
  `ARNDCQEGHILKMFPSTWYV`, mirrored to a symmetric 20x20.
- Re-fetch: <https://www.genome.jp/dbget-bin/www_bget?aaindex3:BETM990101>
- Build-time checks: the Thr row is zero (the defining property of the reference state), the matrix
  is symmetric, all 400 cells present, range −1.34 (Cys–Cys) to +0.66.
- Reference state: pair-contact, mixed sign, mean −0.057 — the same kind as the bundled `MJ` matrix
  (mean −0.079) and **not** the same kind as `Keskin`/`MJ1996` (all-negative, mean ≈ −3.3). Compare
  like with like.
- Reference: Betancourt MR, Thirumalai D. *Protein Sci.* 1999;8(2):361–369. doi:10.1110/ps.8.2.361
- Loader: `tcren.potential.betancourt()`.

## `q_native_reference.csv` — derived

Native-crystal reference means and standard deviations for the Q interface-quality score, so Q is
defined for a single structure rather than only within a cohort.

- Origin: computed by us over the Canonical2026 native crystals.
- Loader: `tcren.cohort` (`q_score`).

## Kyte–Doolittle hydropathy — published (not a bundled file)

The 20 hydropathy values painted onto the ``phobic`` channel of a surface-topology map. Kept as a
literal in `src/tcren/surface.py` (`KYTE_DOOLITTLE`) rather than a CSV: twenty numbers that have not
changed since 1982.

- Origin: Kyte J, Doolittle RF. *A simple method for displaying the hydropathic character of a
  protein.* J Mol Biol. 1982;157(1):105–132. Table 1, the "hydropathy index" column.
- Independent check: r = +0.84 (n = 20) against the hydrophobicity axis
  `Potential.hydrophobicity_fit()` recovers from `MJ1996_contact_energies.csv` — the alternative
  scale `surface_map(..., scale="mj")` selects. Two unrelated derivations of the same axis, so a
  transcription slip would break the agreement rather than hide in it.
- Loader: `tcren.surface.KYTE_DOOLITTLE`.

## `holdout_model.npz` and `holdout_manifest.csv.gz` — derived frozen model

The bundled model retains the original frozen numerical parameters. The manifest names its
training population; the original metadata records 6,429 positive and 1,155 negative complete
training observations over 31 epitopes. Shared nonlinear preprocessing was trained on both
classes; binder moments and confidence moments use label-selected positives. The tier-1
one-class readouts therefore do not imply label-free or negative-independent training.

- Loader: `tcren.score.holdout_model()`; manifest: `tcren.score.holdout_manifest()`.
- Historical recipe: `tcren features -s <manifest structures> -o hold.tsv`, then
  `tcren fit-holdout --features hold.tsv --manifest src/tcren/data/holdout_manifest.csv.gz -o refit.npz`.
  Always write a separate output: evaluation never overwrites the shipped frozen fit.
- Metadata-only migration, 2026-10-14: corrected reduced spring units, finite-grid rupture scope,
  truncated Betti difference and approximate contact-loss labels. `training_catalogue_digest`
  retains the original fitted catalogue; `catalogue_digest` identifies the corrected interface.
  `tests/assets/score/frozen-metadata-migration.json` records both digests, old/new bundle hashes,
  and the SHA-256 of all five unchanged numeric arrays. No refitting occurred.
- Semantic migration: class-II chain-keyed literature descriptors, constrained shear direction,
  Potts chain-keyed sites/sampling, and corrected structure-selection/annotation/frame boundaries
  can change future descriptor outputs. Array identity establishes parameter preservation only;
  it does not certify equivalence of corrected features, a fresh refit, or scientific results.
  Matched fresh reproduction owns compatibility assessment before publication or release.
- Spring units use the default multiplicity/distance² with distances in Å: reduced stiffness
  Å^-2, force Å^-1, work dimensionless, all with an unspecified energy scale. No SI calibration
  was inferred. Unit/count spring weighting has a different arbitrary scale, documented by the API.

Multiplicity-conditioned contact ranks (18 October 2026)
-------------------------------------------------------
Experimental derived resource, never a raw structural probability table. Inputs are
requested final native-b.parquet and external_geometry.tsv.gz contact observations,
with raw PDB/mmCIF correspondence and DTU 22555672 provenance documented above and
in research archive: CONTACT_PRIOR_SOURCES.md. Native CDR:peptide rank references only;
external antibody:antigen exact-opportunity fallback. The approved native/external
marginal resource remains SHA256
63c141f0a8be2e090cda543193148dbd8b06f937dcaf5f4f428124c77d013f16.
Regenerate: tcren contact-weight-calibrate --external external_geometry.tsv.gz
--native native-b.parquet --resource frozen-contact-prior-c.json -o NEW_RESOURCE.json.
Exact input hashes/command/installed-wheel identity are retained in benchmark
results/contact-weighted-20261018/calibration.RUN.json and calibration.json.
Frozen resource SHA256
c096b37a19f5937ce5d40afa4a0c84d4caf8d438a4dc32203e0774d05ebaf07e.
Scientific definition, native/external counts, estimated dependencies and limits:
research archive: CONTACT_WEIGHTED.md. No assay label, potential coefficient or
predictor confidence is used to estimate these ranks. Prior results are preserved.

## Experimental chemical core — derived resource, 20 October 2026

`chemical-core-count-1` is a separately generated empirical reference dependency,
not a bundled force field or raw experimental energy measurement. Its inputs are
existing native class-I CDR:peptide contact observations and the existing curated
antibody peptide-target contact population, exported as physical heavy-atom pairs
at 5 Å. Preserve the original input source/curation provenance and membership
accounting privately where access requires it; public documentation does not
release those structures, identifiers or contact tables.

Regenerate into a new output with neutral file placeholders:
`tcren contact-chemical-core-fit --native observations.parquet --external curated.parquet -o chemical-core.json -t 0`.
Pin exact input hashes, producer version, physical source counts and membership
counts with the resulting immutable resource; record its hash in each consuming
scoring run. No numerical resource or hash is asserted by this recipe alone.

The potential-specific 19-type map separates carbonyl sites, methionine/cysteine
sulfur and unknown-state histidine ring nitrogen without changing the general
contact classifier. Actual contact counts determine symmetric unordered
chemical enrichment; role-specific residue-conditional endpoint compositions
supply the partition. Unseen W=1 is a neutral core completion, while unmapped or
zero-conditional-support queries explicitly abstain. Mapping, reference measure,
completion and mean-per-edge unit addition are versioned together. Counts are
correlated observations and the pooled mixture can be antibody dominated.

All estimation and scoring definitions belong to the library CLI. No assay
labels, fitted combination/source weights, radial/angular or site fit enters
this experimental recipe. Values are dimensionless composition log ratios, not
physical binding free energies or independent likelihoods; neither transfer
accuracy nor a default-model change follows from the estimator. Full equations,
normalization and support policies: docs/contact-penalties.rst; frozen design:
docs/CHEMICAL_CORE_SPEC.md. Original TCRen and existing candidate resources are retained.


## Frozen task-matched ranking references — 22 October 2026

The bundled epitope-ranking and screen references are derived empirical scale
resources, not experimental energy measurements or coefficient-free models.
Means and scales come from a held-out pool of 729 models over 14 off-panel
epitopes. Query cohorts and labels do not estimate these moments at scoring time.
The screen signs are fixed a priori; saved training-pool ROC values check those
signs and are label-dependent diagnostics, not independently estimated directions.
The TCR confidence weight of 0.8 is an explicit design choice informed by recorded
evaluation, not a prospectively validated learned parameter. Epitope weight 0.5
retains the existing equal-weight readout. Applicability cuts and their reference
populations are stored with the resources; flags report by default.

Private input lineage and the exact producer revision are retained in the
benchmark's `bench/scripts/make_screen_references.py` and its release records.
No rerun or new estimation is authorized by this registry. Reproduction requires
the source-verified held-out training inputs and exclusion ledger, not query data.
The resources remain frozen during release verification. SHA-256 identities:
- `epitope_rank_reference.json`: `c5e008f2bcbea1b9f274ee6c330a66838e6a8d7e1596f9eb0bbe1c57580bb98e`
- `screen_reference_epitope.json`: `a42d1c06d19f41725905b63ed45cec03c1a4899735caa4525e692171a3e26366`
- `screen_reference_tcr.json`: `5c55a250e6a4f10a54fd6fd611307f3f0077fbb732435adc28f9e3084f26a225`

Optional orthogonalization key, 23 October 2026: the `tcren-screen-1` format now admits
`confidence_rho` on a structural component, applied as `(z - rho*z_conf)/sqrt(1 - rho^2)`.
NEITHER BUNDLED REFERENCE CARRIES IT, so the three SHA-256 identities above and every score
computed under them are unchanged; a reference without the key takes the original code path.
The key is a caller-supplied frozen coefficient, an estimated reference dependency.
If interpreted as a correlation, it must describe the same frozen population in which
both standardized variables have unit variance; global standardization does not ensure
unit within-cohort variance or independent evidence. No label is required to estimate it. Any resource that does carry one is an external file with its own hash and
its own provenance record; this registry covers bundled tables only.

Release review correction: the epitope-rank energy mean is stored in the raw
`dPhi_tcr_pep` convention, matching the screen reference and the producer. Its
previous sign-corrected mean was incompatible with `direction * (x - mean) / sd`.
This repairs a serialization convention, not a new fit. Previous resource SHA-256
`815b40dc7147fb9199e9f23915fd719b8787462fd28ed14e26e22ed0ff05ba88` is archived with the manuscript release record. Complete-component
rows change by a constant; rows with different missingness patterns need the new
resource identity and must not be compared as though their old scores were unchanged.

Experimental template gate, 4.2.0: `screen --template-metadata` reads a caller-supplied
native-crystal epitope inventory, recording its SHA-256 rather than bundling a new
antigen set. The requested weight is explicit; uncovered epitopes use confidence
alone. The diagnostic 0.6 weight is evaluation-selected and is not a shipped default.
The three frozen resource hashes above remain unchanged.
