The Backtest Metrics That Lie to You

10 min readQuantParadox research

Every backtest metric can be made to look excellent by an unremarkable strategy under the right conditions. Knowing which conditions is most of the skill in reading a report.

The short answer

Win rate, profit factor and Sharpe ratio all mislead when read without the trade count behind them, and the first numbers worth reading on any backtest are sample size, out-of-sample expectancy per unit of risk, and maximum drawdown.

Why is win rate misleading?

Win rate is misleading because it says nothing about profitability on its own — a strategy winning 90% of the time while occasionally losing twenty times its typical win is a losing strategy with an outstanding-looking statistic.

It is also the metric most easily manipulated by exit design, and the manipulation is trivial. Move your take-profit closer and your win rate rises while your expectancy may well fall. Move your stop further away and the same thing happens more dramatically. Any strategy can be tuned to a high win rate; almost none can be tuned to a high expectancy.

This makes win rate a favourite in marketing material and close to useless in evaluation. When you see one quoted without the average win-to-loss ratio beside it, the omission is usually doing work.

Read it only in combination: win rate, average win, average loss, and trade count. Any three of those without the fourth is an incomplete picture.

Is profit factor a reliable metric?

Profit factor — gross profit divided by gross loss — is more informative than win rate but is dominated by extremes, so a single enormous winner can carry an otherwise mediocre strategy to a respectable figure.

The check that costs thirty seconds: remove the best three trades and recompute. If the profit factor collapses, the strategy is a lottery ticket wearing a suit, and its future depends on catching a rare event again rather than on repeating an ordinary one.

Sample size matters here as much as anywhere. A profit factor of 2.0 across 30 trades is considerably weaker evidence than 1.3 across 800, even though the first number looks twice as good.

It is also period-dependent in ways that are easy to miss. Profit factor computed across a decade that included one exceptional year is a different quantity from profit factor computed year by year and averaged, and the two can point in different directions.

Is Sharpe ratio useful for backtesting?

Sharpe ratio is of limited use on short trading track records, because it was designed for portfolios of periodic returns and is extremely unstable on small samples.

It also penalises upside volatility identically to downside, which is straightforwardly wrong for trading strategies whose whole design is occasional large winners. A trend-following system is punished by Sharpe for doing exactly what it is supposed to do.

Its more serious problem in a backtesting context is that it says nothing about how many variants were tried. A Sharpe ratio selected as the best of hundreds of attempts needs a substantial haircut before it means anything, and there are established adjustments for exactly this — deflated and probabilistic variants of the ratio — which almost no retail tool applies.

Sortino, which considers only downside deviation, is a modest improvement for asymmetric strategies. It inherits the same small-sample instability and the same blindness to trial count.

What backtest metrics should you read first?

Read four numbers before any others, in this order, because each one determines how to interpret the ones after it.

Number of trades. Everything else on the page is conditional on it. A metric computed from 23 trades is a rumour.

Out-of-sample expectancy per trade, expressed in R. The average result per unit of risk, on data the parameters never saw. This is the number that most directly answers the only question that matters.

Maximum drawdown and its duration. The depth tells you about capital requirements; the duration tells you whether you would still have been trading it at the end. A 15% drawdown lasting three weeks and one lasting fourteen months are entirely different experiences.

How many variants were tried. Rarely reported and arguably the most important of the four, because it determines how much of the headline result to believe. If the tool does not tell you, you have to count it yourself, and the count includes the ideas you abandoned.

What makes a good equity curve?

A good equity curve is one whose shape matches what the strategy's design predicts — not one that is smooth, because smoothness is the most easily manufactured property of any curve.

An unusually smooth curve on a directional strategy should increase suspicion rather than confidence. Real trading in real markets is lumpy. Smoothness usually indicates either a small sample, a fitted result, or a strategy taking on hidden risk that has not yet expressed itself — the classic example being a system that quietly holds losers until they come back, which produces a beautiful curve right up until the trade that does not.

Look instead at whether the losing periods are consistent with the strategy's win rate and payoff. A trend-following system should show long flat stretches punctuated by sharp gains. If it does not, the backtest is describing something other than trend-following.

And always read the curve alongside the trade list. The curve is a summary; the trades are the evidence, and summaries are where the awkward details go to be averaged away.

Questions people actually ask

What is a good profit factor for a forex strategy?

Figures above 1.5 are often cited as solid, but the number is close to meaningless without the trade count and without checking whether a few outsized winners produced it. A profit factor of 2.0 across 30 trades is considerably weaker evidence than 1.3 across 800 trades over multiple market regimes.

Is Sharpe ratio useful for backtesting?

It is of limited use on short trading track records. It is unstable on small samples, penalises upside volatility identically to downside, and does not account for how many strategy variants were tested before the reported one was selected. Deflated variants of the ratio exist to address the last point but are rarely implemented in retail tools.

What is the most important backtest metric?

The number of trades, because every other metric on the report is conditional on it. After that, out-of-sample expectancy per unit of risk, maximum drawdown together with its duration, and the number of variants tested before arriving at the reported result.

What is expectancy in trading?

Expectancy is the average result per trade, ideally expressed in multiples of the risk taken. It combines win rate and payoff ratio into one figure that answers what you can expect per trade over many repetitions, which is why it is more informative than either component read separately.

The only backtest that settles it is yours.

Build a strategy from a sentence, paste your own Python, or import your live trade history and have it graded. Five full backtests free, no card, and we'll tell you plainly when the result is indistinguishable from luck.

We publish research and tooling, not trading advice, and we make no claim about future returns. Everything above describes how to test an idea — not a reason to trade one.