Crypto Orthogonal Bar Features — Implementation Journal (batch 5, first 8)
Autonomous loop turning 8 orthogonality-proven research formulas into production ClickHouse columns — each proved bit-exact (≤1e-9) AND put through a 16-stage robustness suite (causality · determinism · batch≡streaming · independent-leg · golden+sha · schema-contract · per-feature challenge) before it lands. Build + test only; never merges, never deploys, never backfills.
Robustness standard — everything needed to implement a feature in opendeviationbar-py
This is the bar for shipping ANY new bar feature here. It combines the proven bar_dispersion_entropy/bar_cecp_velocity template with the robustness practices adopted from the MQL5/fxview loop-ten review. Every feature below walks the SAME 16-stage pipeline and must clear the full definition-of-done before its commit.
The 16-stage pipeline (what each flowchart tracks)
Definition of done — the robustness battery every feature must pass
Gate
What it guarantees
≤1e-9 parity (non-vacuous, ≥2-symbol)
Rust matches the Python answer-key to ~15 decimals over real BTC + ETH windows; the test fails if it silently compared zero rows.
Independent cross-check leg
A SECOND source agrees: FOSS library (randtests/lmoments3), an in-Rust from-scratch brute-force (rrbicov/signflux/gmskew), or exact closed-form (recurr/ehlers) — so the oracle isn't the Rust re-checking its own port.
Causality (no look-ahead)
A future bar can never change a past output (truncated-stream == full-stream at each t, via to_bits).
Determinism + batch≡streaming + checkpoint
Bit-identical recompute; live-streaming equals offline batch; resume-from-checkpoint is continuous — all via NaN-aware to_bits compares.
Scale-invariance on real windows
Rescaling real windows ×2 / ×0.5 leaves the scale-free features unchanged (empirical, not just a synthetic property test).
Crash-stress + range/bounds + NaN/Inf totality
Flat / empty / too-short / NaN / Inf inputs never panic and return a defined Option; outputs stay in range.
Golden snapshot (value + sha-over-bytes)
A committed golden is asserted value-equal AND byte-sha-equal IN the hermetic gate — catches an oracle+kernel lockstep drift a value compare cannot.
Schema-contract (no split-brain)
types.rs ↔ schema.sql ↔ column_comments.py ↔ feature_manifest.toml column name-sets must match — guards the 16-file add-one-side-and-forget surface.
Per-feature Challenge-and-Held
An Attacker+Defender re-run the portability/agnostic audit on EACH feature before it commits — never post-hoc.
sha-gated, fail-closed, never-loosen
Input fixture + committed oracle are sha-pinned and hard-halt on tamper; the ≤1e-9 gate is sacred — DEFER a feature, never relax the tolerance.
Conventions (locked)
Input: the trailing 200 bar closes incl. the current bar; quantize each close round(c·1e8)/1e8 (FixedPoint) before any ln. Derive logret = diff(ln(close)), signs = sign(logret).
Oracle fixture: committed, SHA-pinned BTC + ETH close samples; the reference-library version is pinned.
Kernel:libm:: primitives only (bit-determinism), locked compile-time consts (no knobs), NaN/Inf-total, returns None exactly where the oracle does.
Tolerance: a named const with a compile-time assert!(TOL ≤ 1e-9). Sacred — DEFER a feature, never loosen.
Forward-only: the DB column is DEFAULT NULL (NULL on history, fills on new bars). No backfill, no deploy, no live ClickHouse — the loop is hermetic (code + committed fixtures).
Hard rules
One batch PR to terrylica/opendeviationbar-py, per-feature commits; never merge (human decision).
Never weaken a gate, never --no-verify; a feature that can't reach ≤1e-9 is DEFERRED with its reason, not shipped on a loosened gate.
No AI attribution anywhere. Hourly firings; a feature may span several firings (checkpointed per stage).
Adopted from the MQL5/fxview loop-ten robustness review (R1–R7): front-loaded per-feature adversarial/portability audit, shipped-primitive reuse discipline, the 3-way / brute-force independent leg, batch≡streaming + causality + determinism via to_bits, the schema-contract test, sha-gated fail-closed oracles, and the value+sha golden in the hermetic gate.
Read this first — what this loop is doing
We already proved these 8 features add genuinely-new information (they survived the orthogonality + de-duplication audit in PR #540). This loop turns each research formula into fast, production-grade code that is provably identical to a trusted reference (≤1e-9) AND put through the full robustness suite above before it lands. It builds one feature at a time, never deploys or merges, fires hourly (riding out quota refreshes), and a feature can span several firings — it checkpoints after every stage.
Current phase: Batch-level adversarial sweep on top of the per-feature challenges; real gaps fixed.
The 8 features — pipeline status
Each strip is the 16-stage robustness pipeline. green ✓ = done · amber ● = in progress now · slate = not started.
DONE
bar_categorical_recurrence_rate
card 66 · 16/16 stages
How often the price lands back on an exact level it already visited recently — price-level 'stickiness'.
Independent leg: Exact closed-form on integer tie-counts — an in-Rust brute-force is identical arithmetic; the property battery (RR in [0,1], =0 all-distinct, =1 all-equal, hand 3-tie) is the independent leg. scale_check: tie-preserving (constant rescale preserves exact ties).
Latest
Feature 1/8 (bar_categorical_recurrence_rate) COMPLETE — all 16 pipeline stages green. This fire added the ADR, the recurr:* mise tasks (with the regenerate-and-diff oracle drift guard and an explicit nextest filterset so the robustness + schema-contract gates actually run) and 6 slash-command wrappers; ran recurr:check-full to GREEN over BOTH symbols; and held a per-feature adversarial Attacker+Defender challenge (10 findings, all refuted — the bit-exact estimator survives every quantization / tie-grouping / oracle-independence probe). The batch PR is now open as terrylica/opendeviationbar-py#544 with the per-feature oracle-residual table and Challenge-and-Held section. Next: feature 2/8 — bar_sign_markov_flux (directional-flow asymmetry of the up/flat/down sign sequence).
Accuracy proof: Rust vs reference max diff 0.0 (bit-exact; exact-rational estimator, BTC+ETH, SHA-locked) (gate ≤1e-9).
DONE
bar_sign_markov_flux
card 98 · 16/16 stages
Whether up/flat/down moves cycle in one rotational direction rather than reverse — a directional-flow asymmetry of the sign sequence.
Independent leg: In-Rust from-scratch brute-force: an explicit 3x3 transition-count double-loop recomputed independently of the kernel and asserted == kernel over the fixture (no FOSS analytic null). scale_check: sign-sequence invariant (monotone rescale preserves signs).
Latest
Feature 2/8 (bar_sign_markov_flux — whether up/flat/down moves cycle in one rotational direction rather than reverse) COMPLETE — all 16 stages green. This fire finished it: the <=1e-9 oracle parity gate (bit-exact 0.0 over BTCUSDT+ETHUSDT, SHA-locked) via a new SHARED assert_bar_close_oracle helper that the 6 remaining features will reuse; the in-Rust brute-force independent leg (close-comparison signs == the kernel's ln signs over 10000x2 bars); the ADR; the signflux:* mise tasks + slash commands; signflux:check-full GREEN; and a per-feature Attacker+Defender challenge (11 findings, all refuted, documented the three-tier independence model). Batch PR #544 updated to 2 implemented / 0 deferred. Next: feature 3/8 — bar_ramsey_rothman_bicov_lag1 (time-irreversibility: whether recent returns look different played backwards).
Accuracy proof: Rust vs reference max diff 0.0 (bit-exact; exact-rational transition counts, BTC+ETH, SHA-locked) (gate ≤1e-9).
DONE
bar_ramsey_rothman_bicov_lag1
card 113 · 16/16 stages
Whether recent returns 'look different played backwards' — time-irreversibility (sharp-rise/slow-fall vs the reverse).
Feature 3/8 (bar_ramsey_rothman_bicov_lag1 — time-irreversibility: whether recent returns look different played backwards) COMPLETE — all 16 stages in a single fire. This is the FIRST continuous (non-bit-exact) feature: gamma_{2,1}/sigma^3 uses ln + std + cubing, so it matches the numpy oracle to <=1e-9 rather than bit-for-bit — measured residual 9.5e-14 (BTC) / 1.1e-13 (ETH), ~4 orders under the gate. Built+committed: oracle SSoT + BTC/ETH fixtures, the Rust kernel (libm, de-mean, ddof=0, sigma^3) with a property battery proving time-reversal antisymmetry and scale-freedom, the 7-tuple wiring, the forward-only schema across all 4 sources, the <=1e-9 parity gate (shared helper), an in-Rust brute-force formula guard + the numpy cross-language independent leg, the ADR, the rrbicov:* mise tasks + slash commands, rrbicov:check-full GREEN, and a per-feature Attacker+Defender challenge (13 findings all refuted; documented the summation Wilkinson bound). Batch PR #544 updated to 3 implemented / 0 deferred. Next: feature 4/8 — bar_ehlers_increment_asymmetry (skewness of bar-to-bar price changes; closed-form power sums).
Accuracy proof: Rust vs reference max diff BTC 9.49e-14 / ETH 1.13e-13 (continuous; <=1e-9 with ~4 orders headroom, BTC+ETH, SHA-locked) (gate ≤1e-9).
DONE
bar_ehlers_increment_asymmetry
card 26 · 16/16 stages
Whether up-moves and down-moves are lopsided in size — skewness of the bar-to-bar price changes.
Independent leg: Exact closed-form power sums — a brute-force is identical arithmetic; the property battery (scale-invariance x2/x0.5, odd-negation, =0 symmetric, NaN flat) is the independent leg. scale_check: real-window x2/x0.5 invariance.
Latest
Feature 4/8 (bar_ehlers_increment_asymmetry — whether up-moves and down-moves are lopsided in size) COMPLETE — all 16 stages green; the batch is now HALF done (4/8, 0 deferred). E=sum(d^3)/sum(d^2)^1.5 over raw close first differences (no ln); a continuous estimator but the diffs are exact by Sterbenz so the residual is just ~2 ULP (6.9e-17 BTC / 2.8e-17 ETH) — ~12 orders under the 1e-9 gate. This fire finished the feature: the ADR, ehlers:* mise tasks + slash commands, ehlers:check-full GREEN, and a per-feature Attacker+Defender challenge (10 findings, 9 refuted, 1 worked — added an in-Rust brute-force that computes s2^1.5 as s2*sqrt(s2), a different libm path than the kernel's pow, giving ehlers parity with rrbicov/signflux's dual-implementation defense AND cross-validating the pow path; also clarified the Sterbenz claim). Batch PR #544 updated to 4 implemented / 0 deferred. Next: feature 5/8 — bar_cox_stuart_trend_z (a robust trend test: are later prices systematically higher than earlier ones).
Accuracy proof: Rust vs reference max diff BTC 6.9e-17 / ETH 2.8e-17 (~2 ULP, 12 orders under 1e-9; raw diffs exact by Sterbenz, BTC+ETH, SHA-locked) (gate ≤1e-9).
DONE
bar_cox_stuart_trend_z
card 23 · 16/16 stages
A robust trend test: are later prices in the window systematically higher (or lower) than earlier ones?
Independent leg: FOSS: cran/randtests cox.stuart.test (R, GPL, dev-only via Rscript/rpy2) — TRUE independent. scale_check: rank/pairing invariant under monotone rescale.
Latest
Feature 5/8 (bar_cox_stuart_trend_z — a robust trend test: are later prices systematically higher or lower than earlier ones?) COMPLETE — all 16 stages green; 5/8 done, 0 deferred. The Cox-Stuart sign test, z-standardised: S+ counts how many second-half closes exceed their first-half partner, z=(S+-m/2)/sqrt(m/4); at n=200 that's (S+-50)/5 — an integer-count closed form, so BIT-EXACT vs the numpy oracle (residual 0). This fire built the whole feature: oracle SSoT + BTC/ETH fixtures (caught + fixed a numpy np.float64() repr bug in the generator), the kernel + 7 property tests (z=+/-10 on monotone up/down, z=0 trendless, rank-invariant under positive rescale), the 9-tuple wiring, forward-only schema across all 4 sources, the bit-exact parity gate, an in-Rust brute-force S+ recount, the ADR, the coxstuart:* mise tasks + slash commands, coxstuart:check-full GREEN, and a per-feature Attacker+Defender challenge (7 findings, all refuted, no code change — three-tier independence confirmed; FOSS randtests honestly omitted as non-hermetic + different normalization). Batch PR #544 updated to 5 implemented / 0 deferred. Next: feature 6/8 — bar_groeneveld_meeden_b3_skewness (a robust, outlier-resistant skewness of recent returns around the median).
Accuracy proof: Rust vs reference max diff 0.0 (bit-exact; integer-count z=(S+-50)/5, BTC+ETH, SHA-locked) (gate ≤1e-9).
DONE
bar_groeneveld_meeden_b3_skewness
card 99 · 16/16 stages
A robust, outlier-resistant skewness of recent returns, measured around the median.
Independent leg: In-Rust from-scratch brute-force: independent sort-then-average even-n median + explicit MAD-about-median, asserted == kernel (the even-n median averaging is the independence-critical detail). scale_check: real-window x2/x0.5 invariance.
Latest
Feature 6/8 (bar_groeneveld_meeden_b3_skewness — a robust, outlier-resistant skewness of recent returns measured around the median) COMPLETE — all 16 stages green; 6/8 done, 0 deferred. b3=(mean(x)-median(x))/mean(|x-median(x)|) over logret; a continuous median-based estimator so the gate is <=1e-9 (residual 8.6e-13 BTC / 8.0e-13 ETH) rather than bit-exact. Built the whole feature this fire: oracle SSoT + BTC/ETH fixtures, the kernel (libm log, numpy-matching median, MAD) + 7 property tests, the 10-tuple wiring, forward-only schema across all 4 sources, the <=1e-9 parity gate, an in-Rust brute-force median+MAD leg, ADR, gmskew:* mise tasks + slash commands, gmskew:check-full GREEN, and a per-feature Attacker+Defender challenge (8 findings, 7 refuted, 1 worked: added an explicit even-n median-averaging test + documented why median selection is stable under ln rounding via monotonicity). The schema-contract gate caught an initial omission of all 3 schema sources and forced the fix — exactly its job. Batch PR #544 updated to 6 implemented / 0 deferred. Next: feature 7/8 — bar_l_kurtosis_tau4 (robust, outlier-resistant fat-tailedness of recent returns, via L-moments; FOSS lmoments3 cross-check available).
Accuracy proof: Rust vs reference max diff BTC 8.6e-13 / ETH 8.0e-13 (continuous; <=1e-9, ~3 orders headroom, BTC+ETH, SHA-locked) (gate ≤1e-9).
DONE
bar_l_kurtosis_tau4
card 2 · 16/16 stages
A robust, outlier-resistant 'fat-tailedness' of recent returns, built from L-moments.
Independent leg: FOSS: OpenHydrology/lmoments3 lmom_ratios -> tau4 (GPL, dev-only) — TRUE independent. scale_check: real-window x2/x0.5 invariance (L-moment ratio is scale-free).
Latest
Feature 7/8 bar_l_kurtosis_tau4 COMPLETE: all 16 stages. L-kurtosis tau4=l4/l2 via Hosking-1990 PWM over logret. Oracle <=1e-9 (residual BTC 3.42e-14 / ETH 2.93e-14) with lmoments3 FOSS cross-validation baked into the generator (fail-closed, ~1.4e-14); in-Rust binomial-form brute-force independent leg bit-identical (0.0) to the product-form kernel. Wired 10->11 tuple, 4-source schema-contract green, ADR + mise namespace (6 slash wrappers) added, lkurt:check-full GREEN. Challenge HELD (9 findings: 2 false alarms + 7 refuted, 0 worked). Pushed to PR #544 (now 7 implemented / 0 deferred / 1 pending). Next: feature 8/8 bartels_rank_vn_ratio.
Accuracy proof: Rust vs reference max diff 3.419e-14 / 2.931e-14 (continuous; BTC/ETH, 9801 bars each) (gate ≤1e-9).
DONE
bar_bartels_rank_vn_ratio
card 19 · 16/16 stages
A rank-based test of whether recent returns are randomly ordered or have serial structure (momentum / mean-reversion).
Real-data robustness gap CLOSED + P3 HELD. BLOCKER (real-data rule): the 4 continuous features' scale-invariance used SYNTHETIC windows -> replaced with harness_scale_invariance_continuous_real_windows_both_symbols (624 real BTC/ETH-window rescale checks <=1e-9; removed the 4 synthetic tests). Now broader than the pre-batch dispersion/cecp single-window scale harness (closes the vs-previous footprint gap + manifest T3_scale_invariance_real_windows). Committed 4c4f3009, pushed. P3 batch-terminal (1 Attacker + 1 Defender) HELD: both MERGE-READY on feature merit, zero actionable findings; attacker confirmed remaining synthetic data = legit correctness-anchors/degenerate-edge tests (not robustness properties). challenge_held_done=true. Canonical clippy (--all-targets --all-features) exit 0; 35/35 bar_close tests green. Deny still RED only on pre-existing pyo3 #545 (operator-deferred). Merge-ready on feature merit; NO merge, NO ClickHouse touched.
Accuracy proof: Rust vs reference max diff 0.000e0 (BIT-EXACT; BTC/ETH, 9801 bars each) (gate ≤1e-9).