Survivorship Bias in Backtesting: Why Your Results Are Probably Too Optimistic

Survivorship bias is arguably one of the most insidious traps in backtesting, precisely because it is invisible without a specific effort to check for it. It consists of testing a strategy only on assets that still exist today, silently excluding every company that went bankrupt, was delisted, or merged with another firm during the period studied.

The problem is that companies that disappear are, by construction, often the ones that performed worst. By removing them from the tested sample, one mechanically obtains a group of assets biased upward, and therefore a historical strategy performance that could never actually have been achieved in real time by an investor of that era.

How Survivorship Bias Creeps Into a Backtest

The most frequent source of this bias is the use of financial databases that only keep an index's or a stock universe's current composition, with no reconstitution history. If one tests a strategy on the current components of the CAC 40 over the past twenty years, every company that was part of the index at some point but has since been removed — often after a severe underperformance — is ignored.

The same problem affects investment fund databases: when an underperforming fund closes its doors, it generally stops reporting its performance, and some data providers simply remove it from their history altogether. A fund-selection backtest based on such a database will systematically overstate the average performance of funds available at a given date.

Concrete Historical Examples

The most commonly cited example is that of stock indices themselves: the Dow Jones Industrial Average has had nearly its entire original composition replaced over the decades, struggling companies being replaced by better-performing ones. A naive backtest using only the current index composition to represent the market over a hundred years would significantly overstate the return actually accessible to an investor of that time.

In the mutual fund universe, academic studies have shown that simply including defunct funds in the sample can reduce the measured average annual return by several percentage points compared to a sample containing only survivors — a gap far from negligible over time.

Measured Impact on the Sharpe Ratio and CAGR

To concretely illustrate the scale of the problem, imagine a universe of 100 stocks fifteen years ago, of which 20 have since gone bankrupt or been delisted with a total or near-total loss of invested capital. A backtest keeping only the 80 survivors purely and simply ignores these 20 losses of -100% or close to it, which can turn a real annualized return close to 4% into a measured return of 9% or more on the biased sample.

The effect on the Sharpe ratio is just as marked: by removing extreme loss scenarios from the volatility and average return calculation, the Sharpe ratio measured on the biased sample can end up artificially doubled compared to its actual value, giving a false impression of an exceptional risk-return trade-off.

How to Detect and Correct Survivorship Bias

Use Point-in-Time Data

The best protection consists of using so-called point-in-time or survivorship-bias-free databases, which preserve the exact composition of the investment universe as it was known at each historical date, including companies delisted since. These databases are more expensive and less widespread, but they constitute the benchmark for rigorous backtesting.

Reconstitute Index History

Failing access to complete point-in-time data, it is possible to manually reconstitute an index's composition history from official change announcements available from the index providers themselves, in order to reintegrate companies that have since been removed.

Model a Delisting Return

When a delisted company's price history stops abruptly, it is essential to apply a realistic delisting return assumption rather than simply ignoring the observation. Academic research generally suggests applying a significant average loss, often estimated between -30% and -100% of the last known price depending on the reason for delisting, rather than assuming zero loss or simply removing the stock from the calculation.

The Special Case of Bonds and Commodities

Survivorship bias does not only affect individual stocks. In the bond market, issuers that default disappear from benchmark bond indices, and a backtest that ignores these defaults will systematically overstate the real return of a credit carry strategy, particularly in high-yield segments where the historical default rate is far from negligible.

In commodity and futures markets, a similar phenomenon exists with contracts that are delisted or replaced by new maturities: ignoring contracts abandoned for lack of sufficient liquidity can, here too, bias upward the measured performance of a systematic strategy applied to that universe.

The same logic extends to cryptocurrency backtests: thousands of tokens that were actively traded a few years ago have since become worthless or been delisted from every exchange, and any strategy backtest built solely on tokens that remain listed today inherits the exact same upward bias described throughout this article.

These examples are a reminder that survivorship bias is a general data methodology problem, not a mere curiosity specific to listed stocks. Any financial database that silently removes instruments that have ceased to exist must be treated with the utmost caution before being used for a backtest.

A Danger Compounded by Other Biases

Survivorship bias becomes particularly dangerous when combined with look-ahead bias, which consists of using information in a backtest that was not available at the time of the simulated decision. A typical example is selecting today's current index components, then retroactively testing a strategy on those same stocks over the past ten years, thereby compounding the fact of only knowing these companies because they survived with having selected them using future information.

This combination of biases can turn a strategy that is perfectly mediocre, or even losing under real conditions, into a backtest displaying spectacular results, creating totally unjustified confidence in whoever is about to deploy real capital based on these misleading figures.

A useful sanity check when reviewing someone else's backtest — or your own from months ago — is to ask how many of the tested names would be recognizable today as still trading under the same ticker. If nearly all of them are, on a universe spanning ten or fifteen years, that alone is a reasonable red flag that survivorship bias has crept into the sample.

It is therefore essential, before drawing any conclusion from a backtest, to systematically check the provenance of the data used and ask the following question: could this tested universe of assets actually have been assembled exactly this way, with the information available, at each simulated historical date?

A good practice, finally, is to always explicitly state, in the documentation of any backtest shared with third parties, the precise source of the data used and whether or not it is free of survivorship bias. This methodological transparency, still too rare, allows anyone examining the results to correctly assess their degree of reliability before drawing operational conclusions from them.

Conclusion

Survivorship bias perfectly illustrates why data quality matters at least as much as the sophistication of a trading strategy. A statistically flawless backtest built on a biased universe remains fundamentally misleading, regardless of how much rigor was otherwise applied to its design.

To explore your own strategy hypotheses with particular attention paid to the quality and representativeness of the historical data used, the quantitative analysis tools available at /outils on TrueVerdikt offer an educational framework for understanding and visualizing this type of bias, without ever constituting personalized investment advice.