What is a backtesting journal?
A backtesting journal is a log of every test run during the development of a strategy: the rules as they stood, the data window, the parameters, the result, and what was changed next and why. It is the research record, distinct from the trading journal that records live executions.
The reason it exists is that the number of attempts changes the meaning of the result. A strategy that produced a strong out-of-sample result on the first attempt is a genuinely different piece of evidence from the same result reached on the fortieth variant, and nothing in the final report distinguishes them. Only the log does.
Most traders keep no such record. Parameters get adjusted, the tester gets rerun, and the discarded attempts leave no trace, so the surviving result carries no information about how hard it was to find.
The habit is standard practice in any field that reports statistical results, for exactly this reason, and it is the cheapest single improvement available to most retail research processes.
Why do the failed tests matter more than the successful one?
Failed tests matter because they establish the denominator, and without a denominator a strong result cannot be distinguished from the best of many random ones.
The intuition is straightforward. Test one strategy and get a good out-of-sample result, and that is meaningful evidence. Test two hundred variants and report the best, and you have found the highest of two hundred draws, which will look good even if every variant is worthless. The final report looks identical in both cases.
This is why the count has to be recorded at the time. Reconstructing it afterwards is unreliable in a specific direction: the abandoned attempts that felt like exploration rather than testing get forgotten, and the recalled number is always lower than the real one.
The count also determines how much of an adjustment the result needs. Selection effects grow with the number of things tried, and a result that survives an adjustment for forty attempts is a much stronger claim than one that would not. Our article on whether an edge is real goes through how that adjustment works.
What should you record for each backtest run?
Record the rule set version, the data window and instruments, the parameters, the costs assumed, the result with its trade count, and a one-line statement of what this run was testing and what changed from the previous one.
The rule set version is the field people skip and regret. Results are only comparable across runs if you know which rules produced each one, and a strategy under development changes in small ways constantly. A version number and a diff against the previous version is enough.
Costs assumed belongs in the log because it changes between runs more often than anyone admits. A result produced with a tighter spread assumption than the previous run is not an improvement to the strategy, and without the field, the two runs are indistinguishable a month later.
The one-line intent statement is the highest-value field per character. Written before the run, it forces you to state what would count as failure, which is what makes the difference between testing an idea and looking for a number you like.
Trade count belongs next to every result, always, for the same reason it belongs next to every number anywhere in this discipline.
How does a backtesting journal change how you develop a strategy?
Keeping the log changes the process before it changes any result, because writing down what you are about to test makes an undirected parameter search visibly uncomfortable.
The first effect people notice is that the number of variants drops. When each attempt has to be justified in a line of text, the twentieth parameter tweak stops feeling like work and starts looking like what it is. This is the cheapest overfitting control available, and it operates on the behaviour rather than on the statistics.
The second effect is that failed directions stop being repeated. Without a log, the same abandoned idea gets retried months later, because there is no record of why it was dropped. With one, the research accumulates instead of circling.
The third effect appears when a strategy eventually goes live and behaves differently from its backtest. The log is what makes that difference diagnosable — which assumption, which window, which cost model — and without it the investigation starts from nothing.
How does QuantParadox keep this record for you?
QuantParadox applies the out-of-sample split by default rather than as an option, which addresses the largest single reason a research log is needed in the first place.
The default matters because the untracked-attempts problem is most damaging when in-sample and out-of-sample data are mixed freely. When grading always happens on data the strategy was not shaped on, a promising variant has to survive a genuinely unseen period before it counts, and the cost of having tried many variants becomes visible rather than hidden.
The Proof Engine is aimed at the same question from the other side: not what a strategy would have made, but whether its decisions were any good against history it was never shown. That distinction is what a research log is trying to preserve manually.
None of this removes the need for your own record of what you tried and why. A platform can enforce how each test is graded; it cannot know that the run you kept was the fortieth idea rather than the first, and that number is yours to write down.