Search for the best software for backtesting trading strategies and you will get a list of products, a comparison table, and almost nothing about whether any of them will stop you fooling yourself. That is the wrong order. A backtest is a measurement, and the first thing a business asks about a measurement is not which instrument took it but whether the result means anything. Buy the tool second. Decide what the tool has to prove first.
I am going to set out five questions that separate a backtesting setup that can support a decision from one that just draws attractive curves. Each question has a number attached to it, and the numbers are the reason the question matters. If a product cannot answer all five, it is not the right tool whatever the review scores say, and if a free script can answer all five then you have what you need.
Why the question is usually asked backwards
Roughly 92 messages in my inbox use the words luck or gambling, which is a count of a keyword rather than a survey, but it points at something real. One reader wrote: "When I take a bigger lot size my luck goes on sl hit, and when I take small lot size that day I hit targets." Another put it more bluntly: "Before I was trading, very aggressive behavior, 5 lots 10 lots doesn't work with me. It was a gambling."
Both of those people are describing the same problem, which is having no way to tell the difference between a result and a reason. That is precisely the gap a backtest is supposed to close. A third reader had already worked it out: "Getting the hang of things, I think being new I was trying to rush it and had more of a gamblers mindset. Learning to discipline and better risk management."
A backtesting tool is the instrument that is supposed to move you from the first state to the third. It does that only if it can answer the questions below. Most of the time the thing being sold as backtesting software is a chart replay feature with a profit counter attached, which moves you nowhere at all.
The best software for backtesting trading strategies answers five questions
Here they are in order of how much damage they prevent. One, does it hold enough history. Two, does it record how many variants you tried. Three, does it model what trading actually costs. Four, can it keep data away from you until you have committed. Five, can you get your records back out of it. Everything else, the interface, the scripting language, the price, matters less than any one of these.
One, does it hold enough history
This is the question that quietly disqualifies most setups, and it has an exact answer. The standard error of a Sharpe ratio depends on how many return observations you have, which means you can calculate the minimum history needed before a result is distinguishable from zero at all.
Run it for a system with a genuinely good true Sharpe ratio of 1.0, using daily returns and 252 trading days a year. After one year of data the 95 per cent confidence interval around that Sharpe runs from minus 0.96 to plus 2.96. After two years it is minus 0.39 to plus 2.39. It does not exclude zero until 3.85 years have passed. In other words, a two year backtest of a genuinely profitable system cannot statistically distinguish it from a system with no edge whatsoever.
The weaker the edge, the longer the requirement, and it climbs fast. A true Sharpe of 0.5, which would still be a perfectly respectable system, needs 15.37 years of daily data. A Sharpe of 2.0 needs under a year, but systems with a true Sharpe of 2.0 are considerably rarer than backtests showing one. So the first thing to check in any tool is not a feature list. It is how far back the data goes for the instrument you trade, at the resolution you trade it, and whether that history is clean enough to trust.
Two, does it record how many variants you tried
This is the failure that produces most blown accounts built on apparently solid research, and almost no retail tool addresses it.
Suppose you test a strategy and accept a result as real if it would occur by chance less than 5 per cent of the time. Test one variant and there is a 5 per cent chance of a false winner. Test ten and the chance that at least one looks good by luck alone is 40.1 per cent. Test twenty and it is 64.2 per cent. Test fifty, which is a quiet afternoon of adjusting a moving average length and a stop distance, and it is 92.3 per cent. At a hundred variants it is 99.4 per cent.
This is not a fringe concern. It is the subject of a body of academic work on what is called backtest overfitting, including The Probability of Backtest Overfitting by Bailey, Borwein, Lopez de Prado and Zhu in the Journal of Computational Finance, and the same authors' earlier paper in the Notices of the American Mathematical Society. The practical consequence for your purchase decision is simple. A tool that runs a thousand optimisation passes and reports only the best one is actively dangerous, because it has performed the search that creates the problem and then hidden the evidence. A tool that logs every run, with its parameters and its result, lets you divide by the number of attempts. Our pieces on overfitting and on look ahead bias go further into how the damage happens.
Three, does it model what trading actually costs
A backtest that assumes you trade at the mid price with no spread, no commission and no slippage is not a conservative approximation. It is a different business from the one you are in.
The check is straightforward. Take any result the tool produces and ask what happens to it when you add a realistic cost per trade. A system that survives that is worth more investigation. A system that collapses was never a system, it was a measurement of the spread. The same applies to overnight financing if your positions are held, and to the assumption that your order was filled at all during fast conditions. A tool that lets you set these numbers and rerun the test in one click is doing the job. A tool that hides them in a settings file you never open is not. This also connects to the plainer business question of what your setup costs, since research software is itself a fixed cost that has to earn its keep.
Four, can it keep data away from you
The single most valuable feature in backtesting software is the ability to stop you looking. Once you have seen a period of data and adjusted anything in response to it, that period can no longer test your idea, because your idea now contains it.
In practice this means walk forward testing, a holdout period that the software refuses to show you until you have committed to a parameter set, or at minimum a clear separation between the data you develop on and the data you validate on. Note the implication, because it is uncomfortable. The history requirement from question one is for the validation data alone, so a tool holding four years of history does not give you a four year test, it gives you perhaps a two year test and a two year holdout, and neither half is long enough to clear zero for a moderate edge. Data length and honest testing pull against each other, and most people resolve the tension by quietly dropping the holdout. The survivorship bias problem is a cousin of this one.
Five, can you get your records back out
This is the business owner's question, and it is the one that separates a hobby from an operation. Can you export every trade the backtest generated, with timestamps, parameters and costs, in a format you can open in a spreadsheet five years from now?
You need this for three reasons. You will want to compare the backtest against the live results it was supposed to predict, which is impossible if they live in incompatible formats. You will change software at some point, and a strategy locked inside a proprietary format dies with the subscription. And if trading is a business for you, the research that justified a position is part of your records, in the same way an inventory decision is. Our list of metrics every trading business should track assumes you can get at your own numbers, and a surprising number of tools make that hard on purpose.
How to test any tool in an afternoon
Take whichever product you are considering, including the free ones, and run this sequence. Load the longest history it has for your instrument and write down how many years it is. Implement the simplest version of your idea, with no optimisation at all, and record the result. Add a realistic cost per trade and record it again. Now deliberately run twenty variants and see whether the software shows you all twenty or only the winner. Finally, export everything and open the file somewhere else.
You will know by the end of the afternoon whether the tool can support a decision. Notice that nothing in that sequence is about the interface, and nothing requires you to have found a profitable strategy. It tests the instrument, not the idea, which is the right order and the opposite of how most people buy. A tool that passes is worth paying for, and one that fails is not worth having for free, because testing an automated strategy badly is worse than not testing it at all. A bad test does not leave you uninformed, it leaves you confident.
Get the free REX business plan template, one page covering your risk ceiling, your costs and the review schedule that keeps a record honest. One email, no spam, unsubscribe anytime.
Get the free business plan →Frequently asked questions
What is the best software for backtesting trading strategies for a beginner?
The one that holds the most history for your instrument and lets you export your results, which is often a free scripting environment rather than a paid platform. Judge it on the five questions above rather than on the feature list, because a beginner is the person most likely to be hurt by a tool that only shows the winning run.
How many years of data do I need for a backtest to mean anything?
At a true Sharpe ratio of 1.0 you need 3.85 years of daily returns before a 95 per cent interval excludes zero, and 15.37 years at a Sharpe of 0.5. Those figures are for the validation data alone, so a holdout period pushes the requirement higher still.
Is paid backtesting software better than free?
Not inherently. Paid tools often buy you cleaner data and better cost modelling, which are worth real money. They rarely buy you trial counting or enforced holdouts, which are the two features that prevent the expensive mistakes.
Why does testing more variants make my result worse?
Because each test is another chance for noise to look like an edge. With 50 variants at a 5 per cent threshold there is a 92.3 per cent chance that at least one looks profitable by luck alone, so a winner selected from a large search tells you much less than a single prespecified test.
Does a good backtest mean the strategy will work?
No. It means the strategy has not yet been ruled out, which is a weaker and more useful statement. The purpose of the exercise is to reject bad ideas cheaply, not to confirm good ones.
Should I backtest by hand instead?
Manual replay is useful for learning to read a market and useless for measuring an edge, because you cannot run enough samples and you cannot avoid knowing what came next. Use it for training and use software for measurement.
Where REX fits
REX Trading Signal is a free channel about running a trading account the way you would run a small business, with a risk ceiling, a cost line and a review you actually do. There is nothing sold on a promise of returns here. The free business plan template is one page, and its job is to make you write down the limits before the market asks you about them. The piece everything else hangs off is the one page trading business plan template.
About the author. Rex writes REX Trading Signal. He once spent three weeks optimising a system across a few hundred parameter combinations, took the best one live, and watched it behave exactly like the average of the ones he discarded. He counts his trials now and writes the number down before he starts.
Sources and method. The academic treatment of backtest overfitting referred to here is The Probability of Backtest Overfitting by Bailey, Borwein, Lopez de Prado and Zhu, Journal of Computational Finance, 2016, DOI 10.21314/JCF.2016.322, linked above, and the same authors' Pseudo-Mathematics and Financial Charlatanism, Notices of the American Mathematical Society, 2014, DOI 10.1090/noti1105, which is not linked because the publisher blocks automated requests. Both references were verified through Crossref before citing. Every number on this page is my own calculation: the multiple testing probabilities are one minus 0.95 raised to the number of independent variants, and the history requirements use the standard error of the Sharpe ratio at 252 trading days a year, with the confidence level and all assumptions stated in the text so the work can be repeated or disputed.
This article is educational and is not financial advice. Trading gold and other leveraged products carries a high risk of losing money rapidly, and no entry, stop or target discussed should be treated as a signal. No software product is recommended, endorsed or reviewed here, and the Sharpe ratios used are illustrative assumptions rather than results from any real system or any claim that such results are achievable. Reader quotations are real messages, anonymised and lightly edited, and the inbox count is a keyword count rather than a survey. Decide your own risk, and never trade money you cannot afford to lose.