Scope-expansion options for the future

In plain English. Think of the data pool as the set of "test subjects" our research runs against. Right now we deliberately keep that set small — 6 cryptocurrencies — so each round of analysis stays quick and easy to double-check. But someone, someday, will ask "would this still hold up across more coins?" This page is the pre-cooked answer: three off-the-shelf expansion sizes (small, medium, large), exactly which symbols each adds, how much disk and time each costs, and the precise steps to switch one on. It's a recipe card kept in the drawer so we never have to re-derive the recipe.

What is this?

When the data pool's scope was first locked down (a frozen, written-in-stone definition called the pre-spec — the "S0 pre-registration" of what data we promise to study), two larger versions were also worked out in full and then deliberately set aside. This document preserves that pre-work so that, if we ever want a broader pool, we already know:

• which symbols are eligible and when each one started trading (their effective_start date — the first day a coin has enough real history to be usable);
• the exact bar-count math (how many data "cells" each option produces);
• and a copy-paste set of edit instructions so the change is mechanical, not a research project.

Quick numbers

A "fetch" or "cell" here means one combination of (symbol × threshold × time-slice) — one tile in the giant grid the probe analyzes. A threshold (measured in dbps, where 1 dbps = 0.001%; e.g. 250 dbps = 0.25%) is how big a price move must be to close one bar and open the next. More symbols, more thresholds, or more time-slices all multiply the number of cells.

The three options at a glance

Each option is a bigger "test panel." bigblack is the remote GPU server where the data lives. Disk is storage used there; fetch time is roughly how long it takes to gather the cells.

"Cross-asset-class" just means: does the finding hold not only across coins but also across a fundamentally different market (currencies/metals)? That's the strongest kind of confirmation a pattern can get.

The options in detail

Option A — Original minimal (current default)

This is what's locked in the pre-spec. No action needed — it's already in force.

Substrate 1 — Crypto
  Symbols:    BTCUSDT, ETHUSDT, BNBUSDT, LTCUSDT, ADAUSDT, XRPUSDT
  Thresholds: per-symbol availability (ETH only [100, 250]; others [100, 250, 500, 750])
  Slices:     24
  Subtotal:   528 cells

Substrate 2 — Forex
  Symbols:    EURUSD
  Thresholds: 5 (single threshold)
  Slices:     24
  Subtotal:   24 cells

Grand total: 552 cells, ~2-3 GB, ~12-18 min

A "substrate" is just one slab of the data pool. Note that ETH carries only 2 thresholds while the others carry 4 — that's because each coin only gets the thresholds its real history can reliably support, so cell counts differ per symbol.

Option B — Recommended broader

Adds 4 crypto symbols and 2 forex symbols. Multi-threshold forex is still not included. Every added coin was checked to have started trading early enough (before 2020-04-01) to fill all the time-slices cleanly.

Crypto goes from 528 to 888 cells (+360), forex from 24 to 72 cells (+48), for a grand total of 960 cells, ~4–6 GB, ~20–30 min.

When to use Option B:

• Option A's verdicts are strong and you want to broaden cross-symbol confirmation;
• you want a genuine 3-market cross-asset-class panel (instead of a single forex anecdote);
• compute cost isn't a concern.

Option C — Maximalist

Adds 6 more crypto symbols on top of B and turns on multi-threshold forex (testing currencies at four breakout sizes, not just one).

Crypto cell math (each cell = thresholds × 24 slices):

Forex cell math (3 currency/metal pairs × 4 thresholds × 24 slices):

EURUSD  [5, 10, 25, 50] × 24 = 96
GBPUSD  [5, 10, 25, 50] × 24 = 96
XAGUSD  [5, 10, 25, 50] × 24 = 96
Total forex: 288

Grand total: 1,224 + 288 = 1,512 cells, ~7–12 GB on bigblack, ~30–45 min.

When to use Option C:

• you need the strongest possible "does this stay stable across everything" evidence — e.g. right before a final adoption verdict on a metric or feature;
• you need multi-threshold forex evidence to check whether findings depend on the breakout size on the currency side too;
• 16 coins is the largest realistic crypto panel that still uses only full-history symbols (no half-covered newcomers).

What was deliberately left out

Some symbols were considered and explicitly excluded from all three options — mostly because they have thin or partial history that would make results noisy or messy.

If any of these ever become relevant, they get their own Option D / E / F analysis added to the same document, in the same format.

How you'd actually switch options on

An important design detail: the current model has no fetcher and no manifest. The probe reads each cell live from ClickHouse (the database, running read-only on bigblack) at run time. That makes broadening the scope refreshingly mechanical:

1. Bump the scope in pre-spec.md: add a changelog row and update the symbol × threshold matrix (re-add a forex section if reintroducing currencies).
2. Update the probe's constants in orthogonality_probe.py — SYMBOL_THRESHOLDS, MIN_YEAR / SLICES, THRESHOLD_TIERS. Heads-up: reintroducing forex is new code, not just a constant flip — you'd add a second data path (fxview_cache.forex_bars FINAL plus the column set from the archived forex-availability-probe.md), because the probe is crypto-only today.
3. Re-run the probe — it reads the new cells live (read-only). No fetch step, no manifest to grow, nothing to hand-edit.

There's also no "downgrade" needed: the scope is simply whatever the probe's constants say at run time, so narrowing is just a constant edit.

Provenance: how every verdict stays traceable

Every pre-spec version carries a content hash (a short fingerprint of its exact text). That lets any future result cite precisely which scope it ran against — so reviewers can tell what was and wasn't covered.

"computed on shared_data pre-spec-v1 (hash abc123) covering 552 cells"

"computed on shared_data pre-spec-v2 (hash def456) covering 960 cells, of which 552 inherit from v1"

This gives clean provenance for both metric and feature audits. (The fetch-time version bookkeeping described in the source was for the now-removed parquet model and no longer applies under the live-read model.)

One-glance reference card

A (default):   6 crypto + EURUSD @ 5             = 552 cells
B (broader):   10 crypto + 3 forex @ 5           = 960 cells
C (max):       16 crypto + 3 forex × 4 thresholds = 1,512 cells

A: BTC ETH BNB LTC ADA XRP             | EURUSD@5
B: + DOGE BCH LINK XLM                  | + GBPUSD XAGUSD @5
C: + ATOM TRX ZEC HBAR ETC BTCUSD_PERP  | + multi-threshold [5,10,25,50] on all 3 forex