Question one: where is the out-of-sample split?
Ask which period was used to develop the system and which period was used to grade it, and treat a vague answer as an answer.
This single question eliminates the largest share of claims, because a system tuned on all available history and reported on all available history has produced a description of the past. The equity curve can be immaculate and mean nothing about the future.
The follow-up matters as much: was the split chosen before or after the parameters were fixed? A held-out period examined repeatedly during development has been used for development, whatever it is called in the marketing.
An honest vendor answers this immediately and specifically, because anyone who did the work knows exactly where their split is.
Questions two and three: costs and sample size
Ask what spread, commission and slippage were modelled, and how many trades the result rests on — the two questions with the highest ratio of information to effort.
Costs. A zero-spread or fixed-minimum-spread backtest is not a neutral simplification; it is a systematic overstatement that grows with trade frequency. Ask for the assumption in pips and compare it against a live spread on the same instrument during the sessions the system trades.
Slippage is separately worth asking about, because many testers default to none. A high-frequency system tested without slippage is being credited with fills that do not exist.
Sample size. A 68% win rate over 90 trades and over 9,000 trades are different claims wearing the same number. Ask for the trade count beside every headline figure, and be sceptical of any presentation that shows an equity curve without one.
Question four: how many versions were tested?
Ask how many parameter combinations and variants were tried before this one was chosen, and expect the question to be unwelcome — it is the one that reframes a result as a search.
The best of five hundred variants looks excellent whether or not any edge exists, and nothing about the winner's appearance distinguishes the two cases. There are standard statistical adjustments for this, and they all require the trial count, which is why it is never volunteered.
A vendor who says "we tested a few dozen configurations and here is the deflated figure" is demonstrating more competence than one claiming the first idea worked perfectly.
If the answer is that no search took place, the parameters came from somewhere, and asking where usually produces the search that did happen.
Questions five and six: the data and the fills
Ask which broker's data the test used and how the tester resolved bars containing both the stop and the target — the two mechanical questions that separate a careful build from a hopeful one.
Data source matters because spot forex has no consolidated feed. A test run on one broker's history is an estimate for yours, and the tighter the system's stops, the larger that discrepancy becomes relative to the edge.
Modelling quality matters more than most buyers realise. In MetaTrader terms, a result produced with "open prices only" modelling and a strategy whose stop and target both sit inside one bar is a result whose central assumption was made by the tester's default setting.
Ask for the modelling mode and the tick data source. An honest answer names both; an evasive one usually indicates the test was run at the default and never revisited.
Question seven: what does it refuse to do?
Ask what the system does when conditions are wrong, and treat "it always finds a trade" as a warning rather than a feature.
Every honest system has refusals: periods it does not trade, conditions it sits out, data quality it will not act on. A system with no refusal behaviour is either extraordinarily general or has never been asked the question.
The same applies to the vendor's own claims. Someone who can tell you what their system is bad at — which pairs, which regimes, which months — has measured it. Someone whose system works everywhere has usually only measured where it works.
This question is also the fastest way to identify a martingale or grid system without reading the code: ask what happens after a losing trade. "It recovers" means the position size increases, which means the win rate is high, the losses are rare, and the eventual one is very large.
What about a live track record?
A verified live record is worth more than any backtest, and it needs to be verified in a specific way: published in advance, complete, and covering enough trades to mean something.
Published in advance is the load-bearing requirement. A results page assembled afterwards can show anything, and screenshots prove nothing at all. Third-party verification services that connect to the account directly and publish continuously are a genuinely different standard of evidence.
Completeness matters as much. A record showing one account is a record of the account that survived; ask whether other accounts were run and whether any were stopped, because selective publication requires no dishonesty in any individual post.
And check the length against the strategy's frequency. Six months of a system taking three trades a week is around seventy trades, which is not yet a track record however honestly it was collected.