To: Craig S. Tyle, General Counsel

From: John D. Rea, Chief Economist
Brian K. Reid, Senior Economist

Date: June 30, 2003

Subject: Shortcomings of Traditional Measures of Hedge Fund Performance

Overview
The possibility of making hedge fund investment strategies available to average investors has been the subject of growing interest and speculation in the media. Some of the increased attention has focused on the performance of hedge funds compared with diversified stock portfolios.

This memo points out that hedge fund performance as typically reported tends to overstate the actual performance of hedge funds. Reported hedge fund returns and risks are generally based upon indexes that contain biases: Index returns are biased upward and risk measures are biased downward. Considered together, these biases give the mistaken impression that hedge funds might be suitable for small or unsophisticated investors. In reality, investing in hedge funds requires financial expertise and advanced analytical tools that are typically beyond the reach of typical individual investors and most financial advisors.

The remainder of this memo discusses in greater depth the biases in performance measures, as have been documented in academic research.

Hedge Fund Performance Statistics
Current interest in hedge funds appears to stem from recent reports comparing hedge fund risk-return characteristics to those of diversified stock portfolios. For example, it has been widely reported that hedge funds have produced higher returns with lower risk than the S&P 500 stock index over the past decade and during the bear market.¹ Despite these claims, empirical evidence shows that reported returns on hedge fund indexes are likely biased upward. And equally important, the evidence shows that traditional measures of risk understate the true risk of hedge funds.

Total Return. Hedge fund indexes used to compute total return and other performance statistics contain biases because these indexes are not representative of the universe of hedge funds.² There is currently no universal database that contains records of all hedge funds, both those currently operating and those that have ceased operating. Rather, what is available are several commercial databases that contain subsets of the universe. Hedge funds voluntarily participate in these databases to gain exposure to potential investors. Hedge funds, however, typically provide performance statistics and other information to only one data service, limiting the coverage of hedge funds within any one database.³

Hedge fund indexes have been constructed to track the performance of the funds within each database. Because these indexes only characterize the funds in a given database and not the universe, it cannot be assumed that performance based upon database indexes is representative of all hedge funds. In fact, the database indexes likely overstate the performance of hedge funds. A given hedge fund database tends to contain better performing funds, as poorly performing funds have little incentive to report to the data services. Moreover, a hedge fund database is unlikely to contain a large and representative sample of funds that have ceased operations, a factor that adds to the potential for hedge fund indexes to overstate the returns of hedge funds.⁴

For any given hedge fund index, it is not possible to quantify the magnitude of the bias in its calculated return because quantification requires the universe of hedge funds.⁵ Several researchers, however, have used existing databases to estimate the bias that might occur from failing to have a complete record of the funds that have ceased operations. Based upon this source of bias, these researchers have estimated that hedge fund indexes may overstate hedge fund returns by 2 to 3 percentage points per year.⁶

Not only do the hedge fund indexes have upwardly biased returns, but they also produce wide variation in returns across databases. For the period from January 1995 to April 2001, two researchers found that the annual return for aggregate indexes ranged from a low of 13.6 percent to a high of 18.1 percent.⁷ Similarly, two other researchers found a difference of 1.5 percent per annum over the 1994-99 period between two widely used aggregate indexes.⁸ Furthermore, it has been reported that returns on indexes that appear to represent a common investment strategy were not highly correlated across databases.⁹ Such variation reflects differences in the composition of funds in the databases, differences in methods used to construct the indexes, and differences in the treatment of funds that stopped reporting to the data service.

The upward bias in index returns, along with the variation in returns across databases, makes comparisons of hedge fund returns with returns on other investments problematic. To compound the problem, reliable information for computing hedge fund index returns only begins in 1994.¹⁰ As a result, the history of hedge fund performance spans a period of time too short to fully understand the factors generating hedge fund returns or to generalize about the comparative performance of hedge funds.¹¹ Conclusions that are based on such a limited record simply may not hold for the future.

Risk. Standard deviation is the most commonly used measure of investment risk. Its use is appropriate in those situations in which returns on a security or a portfolio approximate a bell-shaped curve or normal distribution. Standard deviation captures the variability in those returns¹² and, in conjunction with the average return, can be used to calculate the probability of outcomes from investment strategies. A small standard deviation corresponds to a low probability of extreme gains or losses, whereas a large standard deviation corresponds to high probability of extreme gains or losses. The magnitude of the standard deviation can thus serve as an indicator of risk.

Researchers have shown that hedge fund strategies are one instance in which standard deviation is not a sufficient measure of investment risk. Indeed, if relied upon to measure hedge fund risk, standard deviation significantly understates the actual degree of risk.

The most important shortcoming in the use of standard deviation is the failure of hedge fund returns to have bell-shaped, normal distributions. Relative to the bell curve, hedge funds are more likely to have larger losses than would be indicated by the standard deviation of returns.¹³ In addition, the holding of illiquid securities in hedge fund portfolios, a not uncommon occurrence, has been shown to bias the calculated value of a hedge fund's standard deviation downward.¹⁴ This bias, along with the departure from a normal distribution, makes standard deviation a misleading guide to hedge fund risks and could lead to inappropriate allocations to hedge funds in trying to create an overall balanced investment portfolio.¹⁵

Hedge funds contain other important sources of risk that are not captured by standard deviation. First, the lack of transparency in the management of hedge funds presents a significant risk to investors. Most hedge fund strategies are dynamic, meaning that funds can change investment strategies in response to market developments. For example, a hedge fund might go from a net long position in U.S. stocks to a net short position in foreign stocks and bonds as a result of global developments. With the lack of transparency, hedge fund investors typically are not aware of the shift in strategy that by itself could alter the overall risk characteristics of their portfolios. Moreover, even if investors were cognizant of a change in strategy, restrictions on withdrawing investments would often prevent them from restructuring portfolios.¹⁶

Second, leverage and illiquidity are additional sources of risk for hedge funds. Most hedge funds use leverage,¹⁷ which can magnify the fund's gains and losses. Losses not covered with sufficient capital can trigger margin calls that, in turn, can lead to forced liquidation of positions. If portfolio securities are illiquid, as is often the case with hedge funds, forced liquidation can produce substantial losses. The lack of transparency in hedge funds makes it difficult for investors to assess the extent of their funds' exposure to leverage and liquidity risks.¹⁸

The final risk arises from diversification, that is, the holding of a portfolio of hedge funds rather than a single hedge fund. A hedge fund typically has a proprietary trading strategy that, in many instances, is a significant source of fund-specific risk. This risk would point toward diversifying either by holding several hedge funds with distinct strategies or by investing in a diversified fund of hedge funds.¹⁹ Indeed, the empirical evidence confirms that a portfolio of hedge funds can have a lower standard deviation than a single fund.²⁰ The lower standard deviation, however, comes at a cost: A portfolio of hedge funds has returns that are more skewed toward losses and more correlated with the stock market than returns of single funds.²¹

These considerations point to hedge funds being significantly more risky than is commonly portrayed. Investing in hedge funds is a complicated task that requires financial acumen and advanced analytical tools. To complicate the task further, empirical research has found that combining hedge funds with traditional stocks and bonds can change the risk profile of these portfolios by increasing the probability of large losses.²² Given the current state of knowledge, hedge funds or hedge fund investment strategies would not seem to be warranted for average individual investors.

¹ Erin E. Arvedlund, "Let the Masses In," Barron's, May 19, 2003, at F2 and Ken Hoover, "SEC Sharpens Shears for Hedge Funds," Investor's Business Daily, May 29, 2003, at A10.

² For a discussion of the biases in hedge fund indexes, see S. J. Brown, W.N. Goetzmann, and R. G. Ibbotson, "Offshore Hedge Funds: Survival and Performance 1989-95," Journal of Business, 72 (January 1999), at 100-104 and William Fung and David A. Hsieh, "Hedge-Fund Benchmarks: Information Content and Biases," Financial Analysts Journal, 58 (January/ February 2002), at 22-34. For a discussion of the difficulty in developing a benchmark index for hedge fund returns, see Robert Clow, "The Benchmark Index: Is It Just a Pipedream?" Financial Times, Special Report on Hedge Funds, June 16, 2003, at 3.

³ Bing Liang, "Hedge Funds: The Living and the Dead," Journal of Financial and Quantitative Analysis, 35 (September 2000), at 317.

⁴ Maintaining records on funds that have ceased reporting to the data service is extremely important, as the percentage of funds leaving databases is high. For example, annual attrition among funds in one database ranged from 9.1 to 12.3 percent between 1998 and 2001. (Gaurav S. Amin and Harry M. Kat, "Welcome to the Dark Side: Hedge Fund Attrition and Survivorship Bias over the Period 1994-2001, ISMA Centre, Discussion Papers in Finance, The University of Reading, 2002-02, January 29, 2002, at 3-7.) At a 10 percent annual rate of attrition, roughly 40 percent of the funds in a given year would drop from the database after five years. Hedge funds may stop reporting to a database because they have poor performance, have closed to new investors, or have ceased operations.

⁵ Fung and Hsieh, "Hedge-Fund Benchmarks," at 24.

⁶ Fung and Hsieh, "Hedge-Fund Benchmarks," at 24; Amin and Kat, "Welcome to the Dark Side," at 9-11; Liang, "Hedge Funds," at 310; Brown, Goetzmann, and Ibbotson, "Offshore Hedge Funds," at 103.

⁷ Chris Brooks and Harry M. Kat, "The Statistical Properties of Hedge Fund Index Returns and Their Implications for Investors," ISMA Centre Discussion Paper in Finance, The University of Reading, 2001-09, November 10, 2001, at 24.

⁸ Fung and Hsieh, "Hedge-Fund Benchmarks," at 26.

⁹ Harry M. Kat, "10 Things That Investors Should Know About Hedge Funds," Journal of Wealth Management, (Spring, 2003) at 74-75.

¹⁰ Fung and Hsieh, "Hedge-Fund Benchmarks," at 25-26 and Clifford Asness, Robert Krail, and John Liew, "Do Hedge Funds Hedge?" The Journal of Portfolio Management, 28 (Fall 2001), at 6-7.

¹¹ Harry M. Kat, "10 Things That Investors Should Know," at 73.

¹² Variation is measured based on average or mean return.

¹³ Brooks and Kat, "The Statistical Properties of Hedge Fund Index Returns," at 7 and Andrew W. Lo, "Risk Management for Hedge Funds: Introduction and Overview," MIT Sloan School of Business, memo, June 7, 2001, at 11-17.

¹⁴ Mila Getmansky, Andrew W. Lo, and Igor Makarov, "An Econometric Model of Serial Correlation and Illiquidity in Hedge Fund Returns," MIT Sloan School of Business, memo, March 1, 2003, at 7.

¹⁵ Brooks and Kat, "The Statistical Properties of Hedge Fund Index Returns," at 15-18, show that the traditional application of mean-variance analysis to asset allocation of stocks, bonds, and hedge funds could place 50 to 75 percent of portfolio assets in hedge funds. For comparison, high net-worth individuals, the principal investors in hedge funds, had 3 percent of financial assets in hedge fund at the end of 2001. (Michael L. Goldstein and Janai Haynes, The Future of the Money Management Industry 2003, Empirical Research Partners, New York, February 2003, at 3.)

¹⁶ Andrew W. Lo, "Risk Management for Hedge Funds," at 11-17.

¹⁷ Goldstein and Haynes, The Future of the Money Management Industry, at 128.

¹⁸ Lo, "Risk Management for Hedge Funds," at 23-30. Lo also identifies another risk as occurring when prices that had been uncorrelated become synchronized. This is a low probability event akin to a market crash that is almost impossible to identify or detect in past data or standard risk measures without an explicit model.

¹⁹ The relatively short lifespan of hedge funds also argues for diversification. See note 4 and Goldstein and Haynes, The Future of the Money Management Industry, at 124, who report that the expected lifespan is about five years.

²⁰ Fung and Hsieh, "Hedge-Fund Benchmarks," at 28-30; Guarav Amin and Harry M. Kat, "Portfolios of Hedge Funds: What Investors Really Invest In," ISMA Discussion Papers in Finance, The University of Reading, March 18, 2002, No. 2002-07, at 13. It may require, however, at least 15 to 20 funds to be adequately diversified.

²¹ Kat, "10 Things That Investors Should Know," at 78-79.

²² Kat, "10 Things That Investors Should Know," at 79-80.