How long should a backtest be?
"Use at least five years of data" is the advice almost everyone gets, and it answers the wrong question. The length a backtest needs depends on the Sharpe ratio it produces and on how many parameter combinations were tried before that Sharpe was kept. Change either one and the answer changes with it.
How long should a backtest be?
Long enough to satisfy two separate requirements, not one. First, enough years for the Sharpe ratio to be statistically distinguishable from zero at a chosen confidence level. Second, enough years that the same Sharpe still clears the corrected bar for however many parameter combinations were tried before it was kept. Most backtests meet the first and never check the second.
Two different clocks, not one
Minimum track record length answers the first requirement on its own: how many years a given Sharpe needs before it beats a benchmark of zero at 95 percent confidence, assuming exactly one rule was tested. That assumption is doing a lot of work. The moment a parameter sweep enters, the relevant question is not "is this Sharpe real" but "is this Sharpe real given how many others were tried and discarded to get it," which is the same correctionhaircutting a Sharpeapplies to the ratio itself, solved here for years instead.
Both clocks run off the same t-statistic. A record's t-statistic is its Sharpe times the square root of its length in years; the corrected bar it has to clear rises with the trial count. Fix the bar and solve for years instead of for Sharpe, and the result is how long the backtest has to be.
The years a Sharpe needs, by trial count
Computed at 95 percent confidence against a zero benchmark, Bonferroni-corrected, when this page is built. Read down a column at a fixed Sharpe to see what more trials cost in years; read across a row at a fixed trial count to see what a higher Sharpe buys back.
| Annual Sharpe | 1 trial | 10 trials | 100 trials | 1,000 trials |
|---|---|---|---|---|
| 0.50 | 15.4y | 31.5y | 48.5y | 65.8y |
| 1.00 | 3.8y | 7.9y | 12.1y | 16.4y |
| 1.50 | 1.7y | 3.5y | 5.4y | 7.3y |
| 2.00 | 1.0y | 2.0y | 3.0y | 4.1y |
| 3.00 | 0.4y | 0.9y | 1.3y | 1.8y |
At a Sharpe of 1.0, a single untuned rule needs about 3.8years. The identical Sharpe, arrived at after a thousand-combination sweep, needs about 16.4 years, roughly 4.3× longer for the same number on the page. The sweep did not make the strategy worse; it made the bar it has to clear higher.
The uncomfortable consequence
A grid search of fifty fast lengths by fifty slow lengths is 2,500 trials, whether or not the tool ever displays that number. At the far right column, even a Sharpe of 2.0 needs more than four years before it clears the corrected bar, and the retail-typical Sharpe of 0.5 to 1.0 needs several decades. Almost no retail backtest runs that long, which means almost no backtest produced by a real parameter sweep has actually earned the confidence its Sharpe ratio implies.
Assessed from vendor documentation on ourpublished rubric.
What to do with a backtest that's too short
Running more history is not always available, so the practical options are to shrink the other side of the equation instead.
- Fix the search space before you start. Fewer parameter combinations means a lower bar to clear, which is the only lever that does not require more data. Seeparameter optimization and overfitting for what an uncounted sweep costs.
- Report the trial-corrected length, not just the record length.A five-year backtest and a stated trial count of one is a very different claim from a five-year backtest after an unstated sweep, even when the Sharpe printed on both is identical.
- Treat a short, heavily-tuned result as unresolved, not as negative.Failing this test does not mean the edge is fake, only that the available history cannot yet tell the difference between a real edge and the best of however many were tried. Seebacktest overfitting for the other tests that can still say something useful at a shorter length.
Frequently asked questions
How long should a backtest be?
Long enough to satisfy two separate requirements, not one. First, enough years for the Sharpe ratio to be statistically distinguishable from zero at a chosen confidence level. Second, enough years that the same Sharpe still clears the corrected bar for however many parameter combinations were tried before it was kept. Most backtests meet the first and never check the second.
Does a longer backtest always beat a shorter one?
Only if the trial count stays fixed while the years increase. A ten-year backtest that gets re-optimized once a year effectively ran ten trials, not one, which eats back a large share of the extra length. Years and trials move the requirement in opposite directions, and only one of them is usually tracked.
Does this replace minimum track record length?
No, it extends it. Minimum track record length is this same calculation at a trial count of one: the length a single, untuned Sharpe needs to clear a zero benchmark. The moment a parameter sweep enters the picture, that figure is a floor rather than an answer, and the trial-corrected length in the table below is the one that matters.
What if I never counted how many parameter combinations I tried?
Use the honest higher estimate: grid size multiplied by every grid you ran, including the ones abandoned after a quick look. Undercounting trials is the most common way this calculation gets quietly defeated, and the required length grows slowly with the trial count, so a rough overcount costs little.
Do any backtesting platforms compute this for you?
2 of the 20 platforms assessed here apply any multiple-testing correction at all, which is the input this calculation needs: Minerva, VectorBT. None of them report it as a required number of years, which is why the table on this page exists.
Related
Educational content only, not investment advice. The table describes a published statistical relationship and is not a judgement on any specific strategy.