Why Your Backtest Doesn't Match Live Trading

12 min readQuantParadox research

The strategy made 60% a year in the tester and has lost money for three months. Almost everyone's first instinct is to adjust the strategy, and it is almost always the wrong first move.

The short answer

A backtest and live trading diverge for six identifiable reasons — overfitting, unmodelled costs, same-bar ambiguity, data differences, execution latency and changed behaviour — and diagnosing which one applies is possible before making any change to the strategy.

How big a gap is normal?

Some gap is expected and unavoidable: live results are drawn from a different sample than the backtest, so even a perfectly valid strategy will diverge simply through variance over a few dozen trades.

The way to tell ordinary variance from a real problem is to ask whether the live results fall inside the distribution the backtest implies. Resample the backtested trades into thousands of sequences of the same length as your live run, and see where your actual result sits in that distribution.

If live sits at the 30th percentile, nothing has gone wrong — that is a slightly unlucky stretch, and reacting to it by changing the strategy is how a working system gets broken.

If live sits below the 1st percentile of what the backtest could plausibly produce, something structural is different, and the rest of this article is about finding it rather than guessing.

Cause one: the backtest was fitted

The most common single cause, by a wide margin, is that the parameters were chosen using the same data that produced the headline result, so the result describes that data rather than the strategy.

The tell is a clean one. If the strategy was tuned on the whole history and has never been graded on a period it did not see, the backtest was never evidence about the future in the first place, and the live results are the first real measurement.

The related and subtler version is search. Thirty variants tested and the best one traded is a maximum, not a sample, and the expected shortfall between the best of thirty and the truth is large enough to explain most disappointments on its own.

The diagnostic is to grade the strategy once, unchanged, on a period that was never used — and to accept that number as the estimate. If no such period exists, the honest answer is that the strategy has not been validated, whatever its backtest says.

Cause two: costs, and cause three: the ambiguous bar

Costs and same-bar ambiguity are the two mechanical causes, and both are checkable in an afternoon without touching the strategy.

Costs. Compare the spread your tester assumed against the spread your broker actually charged, per session and around news. A strategy targeting 10 pips with a modelled 0.4 pip spread and a real average of 1.3 is losing nearly a pip of edge on every trade, which for a marginal strategy is the entire difference.

Slippage is the half people skip. Market orders fill at the price available, not the price requested, and the difference is systematically worse during exactly the fast conditions many strategies trade. If your tester assumes zero slippage, it is modelling a market that does not exist.

The ambiguous bar. When a single bar contains both your stop and your target, the tester has to decide which came first and the bar cannot tell it. Testers differ, most do not document the choice, and for tight-stop strategies this single assumption can invert the result. Build a deliberately ambiguous case and see what your tool reports.

Cause four: the data is not the same data

Your broker's price series and your tester's are not identical, and the difference matters most for exactly the strategies that are most sensitive to it.

Spot forex has no central exchange, so every broker's feed is its own aggregation. Highs and lows differ by a pip or two, which is irrelevant for a wide-stop swing strategy and decisive for one whose stop sits three pips from entry.

Session boundaries and daily candle closes vary by broker time zone. A strategy referencing the previous day's high is referencing a different price depending on where the day was cut, and that difference does not average out — it changes which level is being traded.

Weekend gaps, holiday sessions and how a broker fills a gap over a stop are all further divergences. None of them are anyone's fault; all of them mean a backtest run on one feed is an estimate for another.

Cause five and six: latency, and the trader

The last two causes sit outside the strategy entirely — one in the wiring, one in the person — and both are commonly misattributed to the strategy itself.

Latency and execution. A backtest enters at the signal bar's close. Live, there is a delay between the bar closing, the signal computing, the order sending and the fill returning. For a daily strategy that is nothing; for one trading M1, it is a meaningful share of the move being captured. Requotes, rejected orders and partial fills add to it.

The trader. This is the cause nobody wants and it is extremely common: the live results diverge because the strategy that was tested is not the strategy that was traded. A skipped setup that looked wrong, a stop moved to breakeven early, an extra trade taken out of boredom — each is invisible in the equity curve and fully explains a gap.

The diagnostic is a trade-by-trade reconciliation: list what the rules would have taken over the live period and compare it against what was actually taken. Discrepancies are the answer, and finding them is far more useful than any change to the rules would have been.

What should you do about it?

Diagnose before adjusting, because every one of the six causes has a different fix and five of them are made worse by changing the strategy.

Work in this order, cheapest first: reconcile the trades you actually took against the trades the rules specify; compare your tester's cost assumptions against your broker's real fills; check how the tester resolves an ambiguous bar; then re-grade the strategy once on data it never saw.

Only after those four does it make sense to consider whether the edge itself has decayed — and that is a claim requiring evidence over a meaningful number of trades, not a conclusion drawn from a bad month.

The failure mode this ordering prevents is the common one: adjusting parameters to fit the recent losing period, which is fitting to noise and produces a strategy that is now overfitted to two datasets instead of one.

Questions people actually ask

How many live trades before I judge a strategy?

Enough that a bad run is distinguishable from a broken edge, which for typical retail expectancies means well into the hundreds. A more useful framing than a fixed count is to resample the backtested trades into sequences the length of your live run and see whether the live result falls inside that distribution — that answers the question directly rather than by rule of thumb.

Should I re-optimise the strategy when live results disappoint?

Re-optimising on the period that disappointed you is fitting to the most recent noise, and it produces a strategy tuned to two samples instead of one while feeling like diligence. Diagnose the cause first — the mechanical ones are checkable in an afternoon and are more often the answer than a decayed edge is.

Is demo trading a fair test of a strategy?

It is a fair test of the rules and a poor test of execution and of yourself. Demo fills are typically better than live ones, there is no slippage pressure, and the psychological conditions that cause traders to deviate from their rules are entirely absent. It is genuinely useful for confirming a strategy does what you think it does, and it will not tell you what your fills will cost.

The only backtest that settles it is yours.

Build a strategy from a sentence, paste your own Python, or import your live trade history and have it graded. Five full backtests free, no card, and we'll tell you plainly when the result is indistinguishable from luck.

We publish research and tooling, not trading advice, and we make no claim about future returns. Everything above describes how to test an idea — not a reason to trade one.