Is your strategy real, or just lucky?
We check your trades against 1,000 coin-flip versions of them. You get a plain verdict, the why, and what to do next.
- Your file is never stored. It is read in memory and dropped.
- Same file, same answer. Every time, for everyone.
- Open source. You can check every line of the maths.
Which columns we used
Drop the backtest of the same strategy. We'll check if your live trades do what the backtest promised.
These move together
If this kept you from trading a lucky strategy, a coffee keeps edge or luck going.
How it judges
Luck can make almost any short record look good. So we ask one question: how often would pure luck do as well with your own trades?
-
We flip a coin for each of your trades.
Same trades, same sizes, same order. Only whether each one wins or loses is left to chance.
-
We do it a thousand times.
That draws everything luck alone could have done with your trades. Then we see where your real line lands.
-
We say “real” only when luck almost never does as well.
And we raise the bar for every other version you tried, because trying many makes one look good by chance.
How we know (the maths)
The range shown for your average trade is an anytime-valid confidence sequence (Waudby-Smith and Ramdas). From 30 trades on, it stays honest even if you check after every trade, so watching it does not fool you. Below 30 trades we show no range: a spread measured on a handful of trades is too unsteady to promise anything.
The strength of the evidence is an e-value. When you upload several strategies, or tell us how many versions you tried, we control false discoveries with the e-BH procedure (Wang and Ramdas) at 5%. Versions you didn't show count as zero evidence.
Two corrections guard the cases that fool simple tests. When trades move together, the spread is widened. When losses are rarer but bigger than wins, the bar for “real” is raised (Hall's Edgeworth correction), and nothing is called real before five losing trades.
The coin-flip picture keeps your exact trades and randomises only their signs. It shows what luck does with your trade sizes, which is what matters.
Every step, with the computer proof behind it, and where I can be wrong
How often it's wrong
A referee should show its own mistakes. These numbers come from the test suite in the repository, and you can run it yourself.
- The measured error rates appear here once the test suite has run on this version.
What it can't tell you
An honest answer includes its limits.
- It can't see your code.If your backtest peeks at future prices or skips fees, the trades look better than they were. When a record looks too good, we flag it and give you a prompt to have your code checked.
- It can't promise the future.A real edge can stop working. We check if yours has faded, and you can come back to re-check.
- It needs at least 30 closed trades, and five losing ones.Below that, luck can make almost any record, and an unseen loss could be any size. You'll get “too early” or “can't tell” instead of a guess.
- It treats trades as separate events.Trades that depend on each other, like many open at once, can make the evidence look a little stronger than it is.
Also works inside your AI assistant
If an AI writes your bot, it can run the same judge on your backtest and give you the same honest answer, with the reasons. Add it once.