Which slices of the data are actually empty — and the rules for working around them
A running log of holes discovered in the crypto price-bar database that the research pool reads from. Some symbol / setting / time-window combinations that we expected to contain data turn out to be empty, and this page records exactly which ones, with the proof and the workaround rules.
In plain English. Think of our research data as a giant spreadsheet: one row per coin, one column per time window, with a number in every box (a "box" is what we call a cell). The plan said every box should be full. When we actually looked, a handful of boxes are blank — a coin was never recorded at a certain sensitivity, or its history doesn't go back as far as we assumed. This page is the honest list of which boxes are blank, so nobody later mistakes "we have no data here" for "the answer here is zero."
What is this?
Two documents describe the data pool. One, pre-spec.md, is locked — it states the intended scope and can never be edited (it's frozen with a content fingerprint). The other is this finding, which is mutable: it accumulates real-world discoveries over time. They're kept separate on purpose — intentions don't change, but reality keeps revealing new gaps.
The findings come from probes — short, read-only database queries (no writing, no editing) run against ClickHouse, the high-speed database on the bigblack server that stores the price bars. The golden rule: a finding here describes the actual state of the data; it never changes the locked plan. Empty boxes surface naturally — when the data is fetched, those boxes simply report bar_count = 0.
A note on the jargon. A "bar" is one price candle (open / high / low / close over a span). dbps is the sensitivity setting that decides how big a price move triggers a new bar — 100 dbps = 0.1%, so a smaller dbps means more bars. A "slice" is a fixed two-month time window. A "cell" is one specific coin × sensitivity × slice combination — one box in the spreadsheet.
The headline numbers
552
cells the original plan (v1) intended
42
cells found empirically empty (7.6%)
510
cells expected to hold usable data
441
cells in the current plan (v2, crypto-only, 2021+)
The plan was later revised from v1 (552 cells) to v2 (441 cells). v2 simply deleted the problem cells from scope rather than leaving them blank, but the v1 accounting is kept for the record.
The four empty patches we found
1 · Litecoin at the finest setting is essentially a ghost
LTCUSDT @ 100 dbps (Litecoin, finest 0.1% sensitivity) has almost no data in the entire table — 31 bars across all of history, versus 7–8 million for the other coins at the same setting. Nearly all of those 31 live in two tiny clusters (26 bars in Sept 2022, 5 in April 2026) that fall between our research windows, so they don't even land in a usable slice.
Best guess at cause: Litecoin was probably never historically rebuilt at this fine resolution — the data-population pipeline prioritized the coarser 250 dbps+ settings for the lower-activity coins. The two tiny clusters look like leftover test artifacts. Not investigated further; out of scope for the data layer.
Verdict: all 24 of the planned Litecoin @ 100 dbps cells report bar_count = 0. Re-confirmed on 2026-05-30 (26 bars in the 2021+ span).
Still fine: Litecoin at 250 / 500 / 750 dbps is fully usable. Treat Litecoin as a 3-setting contributor, not 4.
v2 fix (2026-05-30): Litecoin @ 100 was dropped from scope entirely (LTCUSDT = [250, 500, 750]).
2 · Bitcoin and Ethereum history only goes back to 2021, not 2018
The plan assumed BTCUSDT and ETHUSDT bars start on 2018-01-16. In reality the bars table begins on 2021-01-01 for both. The earlier (2018–2020) bars were simply never built into the current table.
Why the surprise: the plan pulled its start date from symbols.toml, which records the "earliest clean trade data" (2018-01-16). But bars are a derived product built from trades — and the bar-building only goes back to 2021. BTC/ETH are the only pair where these two dates diverge; the other four coins have bars back to their listing.
Impact: the three oldest windows — slice22 / slice23 / slice24, covering 2020-04 → 2020-11 — are empty for both coins. That's 18 cells (BTC × 4 settings × 3 slices = 12, plus ETH × 2 settings × 3 slices = 6).
Still fine: slices 1–21 (2021-01 → 2026-02) are fully covered for both coins.
v2 fix (2026-05-30): this finding drove the decision to drop the 2020 slices for everyone, so all symbols now span 21 slices (2021+).
3 · Ethereum's finest setting stops in mid-2025
ETHUSDT @ 100 dbps has no bars after 2025-06-17 — the data-population stalled there, while the coarser ETH @ 250 dbps stream stayed healthy through 2026-02. This was a new find on 2026-05-30; an earlier 10-window spike missed it because it happened to skip the affected window.
Impact: the three most-recent windows — slice01 (2026-01), slice02 (2025-10), slice03 (2025-07) — are empty for ETH @ 100. 3 cells out of 441.
Still fine: ETH @ 100 covers slice04 onward (back to 2025-04→05); ETH @ 250 is unaffected (all 21 slices). Treat ETH @ 100 as covering 18 of 21 slices.
4 · The 750 dbps setting is too sparse for some math
At the coarsest setting, 750 dbps (0.75%), price rarely moves enough to make many bars, so a two-month window often holds fewer than the 1,000-bar floor that some statistics need to be trustworthy. This isn't an empty cell — it's a "use with care" cell.
Extent: 24 of 441 cells land in the [200, 1000) range. BTCUSDT @ 750 misses the floor in ~10 of 21 slices (worst: slice01 = 242 bars); BNBUSDT @ 750 misses it in ~9 (worst: slice12 = 312). A few 500 dbps cells dip too. Crucially, zero cells fall below the 200-bar minimum, so a candidate feature is always computable where a cell exists at all.
What still works: the simple rank-correlation measure (Spearman — how well two things move together by rank, valid down to 100 bars) stays reliable. The pickier measures (the "shape" stat h_norm, and conditional-dependence measures Chatterjee ξ, CODEC/FOCI, MIC) need ≥1,000 bars and are unreliable for these sparse cells.
The rule: 750 dbps is tagged spearman_only — THRESHOLD_TIERS = {100: full, 250: full, 500: full, 750: spearman_only}. The probe flags each cell meets_floor_1000, and the cross-window aggregation excludes sub-floor cells so a fragile 750 cell can never silently corrupt a verdict. A rejected alternative was a longer 3-month window for 750 only — rejected because it would break the clean, uniform grid.
(Aside) Forex was removed entirely
EURUSD @ 5 dbps was dropped on 2026-05-30 — the pool is crypto-only for now by operator directive. No forex data-quality tracking applies while forex is out of scope; the old availability notes are archived for possible reintroduction.
The full empty-cell tally (v1 accounting)
Source of empty / unusable cells
Cells
Litecoin @ 100 dbps — entirely empty column
24
Bitcoin slices 22/23/24 × 4 settings
12
Ethereum slices 22/23/24 × 2 settings
6
Total empirically empty cells
42
Pre-spec v1 intended cells
552
Expected populated cells
510
42 of 552 = 7.6% of the v1 scope surfaces as empty data files. (The mid-2025 ETH @ 100 truncation and the 750-density issue were catalogued later, against the 441-cell v2 grid.)
The rules everyone downstream must follow
Filter on bar_count >= 100 before using any cell. 100 bars is the Pearson convergence floor — the minimum needed for a basic correlation number to settle down. Empty cells still appear in the index (manifest.json) with their fetch attempt recorded, so the audit trail is complete; they just carry bar_count = 0.
Don't overclaim cross-coin agreement. At 100 dbps, "stable across all symbols" means 5 coins (BTC, ETH, BNB, ADA, XRP), not 6 — Litecoin is absent. For the 2020 windows (slices 22/23/24) it's 4 coins (BNB, LTC, ADA, XRP) — BTC and ETH are absent there.
Count effective windows correctly. BTC and ETH contribute 21 effective slices, not 24. ETH @ 100 specifically contributes 18 of 21.
Trust Spearman, gate the rest at 750. Conditional/shape metrics are off-limits on sub-floor 750 dbps cells; rank correlation stays valid.
Never edit the locked plan or the manifest by hand. Record the empirical fact here; the fetcher owns the manifest and the gaps surface on their own. If a backfill is ever run, the idempotent fetcher picks up the new data automatically and the cells fill in.
Bottom line. About 8% of the originally-planned data boxes are genuinely empty (Litecoin's finest setting, plus 2018–2020 for Bitcoin/Ethereum, plus mid-2025-onward for Ethereum's finest setting), and the coarsest 750 setting is too sparse for the fancier statistics — so always require at least 100 bars per cell and treat 750 as rank-correlation-only before drawing any conclusion.