Haircutting a Sharpe ratio for multiple testing
Harvey and Liu asked what happens to a published finance result once you account for the hundreds of ideas tested and never published. Their answer was a haircut: the portion of a Sharpe ratio that does not survive the corrected bar. Applied to your own parameter sweep, it is usually large.
Why the bar moves
A t-statistic of 2 is the conventional threshold, and it means a result this strong appears by chance about one time in twenty. Test twenty candidates and you should expect one to clear it on luck alone. The correction raises the bar so that the one-in-twenty applies to your whole search rather than to each attempt separately.
The haircut is what that costs you, expressed in the units you started with. Instead of telling someone their p-value is now inadequate, you tell them their Sharpe of 1.2 is really a Sharpe of 0.6, which lands rather differently.
Worked example
A 10-year record with an observed annual Sharpe of 1.2, which gives a t-statistic of 3.79. Under a Bonferroni correction at 5 percent, here is what survives as the number of tested candidates grows. Computed when this page is built.
| Trials | Required t | Surviving Sharpe | Haircut |
|---|---|---|---|
| 1 | 1.96 | 1.20 | 0% |
| 10 | 2.81 | 0.93 | 22% |
| 50 | 3.29 | 0.78 | 35% |
| 200 | 3.66 | 0.66 | 45% |
| 1,000 | 4.06 | 0.54 | 55% |
Bonferroni only, for readability. Holm and the false-discovery-rate corrections are less severe, so treat this column as the conservative end of the range rather than the answer.
Who applies a correction at all
2 of the 20 platforms assessed apply any multiple-testing correction: Minerva, VectorBT. Everywhere else, the trial count exists only in your head, which means the adjustment happens only if you do it deliberately.
Assessed from vendor documentation on ourpublished rubric.
Applying it in five minutes
- Count your trials honestly. Grid size times the number of grids you ran, plus the earlier attempts you would rather not count.
- Compute your t-statistic: annual Sharpe times the square root of years.
- Look up the corrected bar for that trial count, from the table above or from any normal quantile function.
- If your t-statistic does not clear it, the honest surviving Sharpe is zero. Report that rather than the raw figure.
- If it does clear, the gap is your margin, and it is usually thinner than the original number suggested.
The reason to do this before anyone asks is simple: whoever evaluates your strategy next will ask how many things you tried. Having the answer already computed is a stronger position than having to reconstruct it.
Frequently asked questions
What is a haircut Sharpe ratio?
The Sharpe that remains after adjusting the significance bar for how many strategies were tested. Harvey and Liu proposed it for published asset-pricing factors, where hundreds have been tested and only the winners appear in journals. The same logic applies to a parameter sweep on your laptop.
How is it different from the deflated Sharpe ratio?
They are close relatives with different framings. The deflated Sharpe returns a probability that the true Sharpe exceeds a benchmark, folding in track length, skew and kurtosis alongside the trial count. The haircut returns a shrunken Sharpe, which is easier to communicate because it stays in the units people already understand. Use whichever your audience reads more easily; they will not disagree by much.
Which correction should I use?
Bonferroni is the most conservative and needs only the trial count, which makes it the practical choice when you are doing this by hand. Holm is uniformly less conservative than Bonferroni and just as easy to justify. Benjamini, Hochberg and Yekutieli control the false discovery rate instead of the family-wise error rate, which is more appropriate when you expect several genuine winners rather than one. Harvey and Liu report all three and average them.
Does the haircut ever reach 100 percent?
Yes, and that is the useful case. If your observed t-statistic never clears the corrected bar, the surviving Sharpe is zero, meaning the result is indistinguishable from the best of that many random attempts. Seeing a haircut of 100 percent is a cleaner signal than any amount of narrative about why the strategy should work.
What counts as a trial?
Every configuration evaluated, including those discarded immediately, plus the ones you tested weeks earlier on the same data and forgot. If you are unsure, the honest move is to use the higher figure. Undercounting trials is the single most common way this adjustment gets quietly defeated.
Related
Educational content only, not investment advice. The worked example illustrates a published statistical adjustment and is not a judgement on any specific strategy.