Index construction in the hedge fund world has always been greeted with some skepticism by mainstream analysts. First, there is self-selection bias: only the winners report.  Then there is backfill bias: winners include results prior to reporting to an index. Finally, there is survivor bias: when a fund gets into trouble, it ceases to report.

These biases make it troublesome to compare hedge fund results with an index of investable stocks. Hedge fund investors cannot buy past results. And they may not be able to buy successful hedge funds that are closed to new investors. There are some hedge fund index providers – notably MSCI, CFSB/Tremont, HFR and S&P, among others.

Still, investable stock indexes may suffer from similar flaws, albeit in the opposite way. Jeremy Siegel, in his Stocks for the Future, noted that the components of the original 1957 S&P 500 provided a better return than the periodically reconstituted S&P 500.

And that may be true of hedge fund databases too. In an article in the Financial Analysts Journal last year,  “Measurement Biases in Hedge Fund Performance Data: An Update,” William Fung and David A. Hsieh observe that in the “2008 annual ranking by the business publisher Institutional Investor of the top 100 single-manager hedge fund firms … we found about 10 occurrences in which funds (and/or firms) were listed in the graveyard database of either Hedge Fund Research (HFR) or TASS, even though they were actively reporting to the other database vendors.”

Which leads to a new bias, they write: “Existing models for correcting performance measurement biases are unable to detect potential data errors arising from (1) hedge funds that migrate from one database vendor to another and (2) merged databases.”

As the winners of the AIMA Canada- Hillsdale Research Award, Peter Klein, Daryl Purdy and Isaac Schweigert point out : “Fung and Hsieh …  report that 40% of the top 100 single-manager hedge fund firms in the 2008 annual ranking by Institutional Investor are missing from the most popular hedge fund databases, which could represent ‘a sizable bias in the opposite direction – namely, that good performance may also be excluded.’”

Let’s call it a bias against success.