The candidate intake standard: one ledger, one preflight, zero double-evaluation

A single, machine-readable logbook tracks every research idea ("candidate") from the moment someone names it through to a final pass/hold/block verdict — so the same idea is never tested twice and nothing slips through half-checked.

In plain English. Imagine a hospital admissions desk. Every patient gets ONE chart the instant they walk in, and that chart follows them through every room and every test. The chart can only gain new notes — old notes are never erased. Before a patient is sent for an expensive scan, a nurse runs a quick checklist to make sure nobody has already scanned them and that all the paperwork is in order. This finding sets up exactly that system, but for research ideas instead of patients: one chart per idea, one quick preflight checklist, and a hard rule that no idea is ever evaluated twice.

What is this?

Researchers here are constantly proposing "candidates" — new measurements or signals that might add value to the trading-data pipeline. The problem with any busy research effort is that ideas get re-discovered, re-tested, and re-argued, wasting hours each time. This standard fixes that by giving every candidate a permanent home and a fixed path to walk.

One ledger row per candidate. The instant an idea is identified, it gets a single row in a committed file called candidate_ledger.jsonl — the "single source of truth" (SSoT), the one authoritative record everyone trusts. (A .jsonl file is just a plain-text list with one record per line, easy for both people and programs to read.)
Append-only history. The row can only gain notes over time; nothing is overwritten or deleted. You can always see how a decision was reached.
One tool to manage it. A small program, candidate_ledger.py, adds rows, checks for duplicates, and runs the preflight checklist.

The fixed life of a candidate

Every candidate moves through named stages, like a parcel moving through shipping statuses. You can always tell at a glance where any idea stands:

identified → pending → evaluating → evaluated → promoting → promoted

Three of these are terminal — dead ends an idea can land in and never leave:

deferred — set aside for now (may come back later).
banned — ruled out and not allowed back.
out_of_scope — not part of what this effort is studying.

How it works — the standard flow for a new candidate

When a new idea appears (for example, in a fresh crypto-discovery campaign), it walks these eight steps in order. Each step guarantees one specific thing — that guarantee is the whole point of the step.

Step	What you run / produce	What it guarantees (plain English)
1	`candidate_ledger.py --check "<name>"`	No double evaluation. Searches every existing name and nickname ("alias") — both exact matches and partial ones ("containment") — so you never re-test something already on file.
2	`--add` with a number, name, family, and an `--audit-folder`	One row at birth. Creates the single ledger row the moment the idea is identified, and points it at the folder where its evidence will live.
3	one row in `candidate_knobs.jsonl`	Settings declared up front. "Knobs" are the adjustable settings an idea uses. The "family manifest" (the spec sheet for a group of related ideas) fails closed — i.e. it refuses to proceed — if you under-declare them. Fail-closed means "when in doubt, stop," the safe default.
4	`--preflight "name1,name2,…"`	One command replaces a 2–3 hour manual alignment. In a single run it checks for duplicates, confirms the knob row exists, checks the manifest floor, returns the "Axis-2" verdict (the parameterless-design check, explained below), and reports the stage — per candidate. Anything not marked OK/PASS is the exact item to go fix.
5	the evaluation loop runs → one record per "cell"	Group comparisons are mandatory. When ≥2 candidates are tested together (a "slate"), each must emit a `cross_candidate` "Finish" file. Without it, the system caps the result at PENDING-SIBLING instead of a full pass — it won't let a group claim victory without comparing members against each other.
6	`campaign_summary_adapter.py --records <dir> --out …`	Honest scoring, never optimistic. It reads the list of already-promoted features automatically, takes the worst of the stored vs. freshly-recomputed score (fail-closed), enforces a coverage floor (at least 10 data slices over 7 market eras, else it's marked INCOMPLETE), and treats an empty results folder as a hard error.
7	`three_axis_gate.py` with the orthogonality + agnostic inputs	One clear verdict. Produces a single promote / hold / block decision per candidate.
8	`--set-stage cand-XXXX <stage> --note "…"`	The outcome is written back. The ledger records what happened, so the next campaign's Step-1 `--check` can see it and avoid repeating the work.

What "Axis-2" means (Step 4). The research uses three "axes" or lenses to judge a candidate. Axis-2 is the parameterless check: does the idea avoid arbitrary hand-tuned numbers that could be cherry-picked to flatter the result? Preflight runs this verdict early so a candidate with hidden free parameters is caught before any expensive evaluation.

Key numbers and gates

Ledger row per candidate

Steps in the standard flow

Normal stages (+3 terminal)

2–3 h → 1 cmd

Preflight replaces manual alignment

≥2

Slate size needing cross-checks

10 / 7

Coverage floor: slices / eras

How the final verdict is decided

The "adapter" (the tool in Step 6) deliberately leans pessimistic — it would rather under-promote than over-promote a fluke. The final verdict is the worst of three things:

the panel verdict measured against the worst valid cell (the least flattering data slice),
the "sibling band" — how the candidate looks next to the others it was tested with, and
the "leave-one-out band" — a robustness check (does the conclusion survive dropping any single piece of evidence?).

The asymmetric sibling rule

"Siblings" are candidates tested side by side. The demotion rule is deliberately asymmetric — it is lenient toward winners and strict toward losers:

Already-promoted features are NOT demoted just because they're linked to the siblings they beat. They won fairly; a link to a weaker rival doesn't drag them down.
Deferred candidates ARE demoted when linked to a promoted feature (this is "panel staleness" — the comparison set has moved on), and also when they clash with each other via "mutual-exclusion" conflicts (two ideas that can't both be true).

What we admit we haven't solved yet (honest limits)

The finding is candid about two known gaps:

Panel staleness beyond the immediate slate. Right now the per-cell records don't carry the full list (and a "hash" — a short fingerprint) of the comparison columns. So the adapter can spot a rival promoted in the same slate, but not a panel that quietly grew afterward. Going-forward requirement: every per-cell record must store the panel's column list plus a panel hash, so any growth is detectable.
Family manifests only exist for researched families. Spec sheets currently exist only for the families already studied — mfdfa, ordinal, and chaos_embedding. A brand-new family needs its one-block "floor" definition before its verdicts can be trusted — and the preflight step flags this gap for you.

Bottom line. Every research candidate now lives in one append-only ledger row and passes a single preflight command that catches duplicates, missing settings, and free parameters up front — so the team never evaluates the same idea twice and every promote/hold/block verdict is honest and traceable.

Crypto Shared Data Pool · finding · source: findings/evolution/shared_data/candidate-intake-standard.md · ← back to the pool · 2026-06-16