Why Don't More People Use Index Funds?

Here at The Index Investor, we are strongly committed to the twin disciplines of adequate diversification across and within asset classes, its implementation through low-cost index funds. Each month we try to present more evidence in support of these views, and, on those rare occasions when it shows up, contrary evidence as well. Over time, we believe that the weight of evidence that has been accumulated strongly supports our basic views.

Unfortunately, convincing though it may be, that body of evidence has to be set against a still very large and thriving active investment management industry. While we don't dispute the statistical probability that some of these active managers indeed possess a combination of superior information and analytical models that enable them to consistently earn "above the benchmark" returns, we very strongly believe that the number of such managers is well below the total number of active fund managers. Logically, this compels us to ask why more people aren't using index funds.

We have thought a great deal about this subject. When you consider the financial consequences of people having poorly diversified portfolios and paying too much for active management, one quickly concludes that this is a critical issue that too few people have directly addressed. Just think of the amount of resources that will NOT be available to finance people's retirements as a result of their poor investment decisions, and the reduced quality of life that will result. On the one hand, its depressing; on the other, it motivates you to try to change things…

So, back to our question. Why don't more people adequately diversify their portfolios, and invest them in index funds? Our answers to these questions fall into three broad categories, which we term "rational explanations", "information and herding" and "irrational causes".

Rational Explanations

The first rational explanation is that given the size and timing of their financial goals, and their current and expected savings, many investors have no other choice but to "swing for the fences". In other words, their only hope of achieving the goals they have set is to earn the kind of exceptionally high returns that can only come from concentrating one's investments in a limited number of securities in the hope of earning very high returns on them over a relatively short period of time. In support of this potential explanation, one can find a number of studies that have concluded that the majority of Americans have not saved enough money to achieve the standard of living they aspire to in their retirement years. Other studies have shown how many American investors' portfolios are relatively undiversified. Set against this evidence, however, are an equally impressive list of studies that conclude that many Americans have not thought systematically about their retirement savings needs and how to satisfy them. Absent such consideration, one cannot logically decide to concentrate one's investments in a limited number of securities if one's goal is to adequately finance one's retirement income needs.

This leads us to our second rational explanation for the investor behavior we observe. Could it be that active investment management provides people with some additional non-financial benefits that more than make up for the apparent financial disadvantages of this approach (compared to indexing)? When we thought more along these lines, we developed a hypothesis that the non-financial benefit in question might be the "social capital" or "reputational benefits" that accrue to people who others perceive to know a lot about and/or "be successful at investing". We have not been able to find any studies that have directly tested this hypothesis; however, we have found some work that has supported similar hypotheses in other areas. As a result, we believe that this is definitely an area that merits more in-depth study.

Information and Herding

Perhaps the simplest explanation for the continued popularity of active investment management is that many investors have not yet heard and/or understood the arguments in favor of diversification and indexing. To be sure, there are strong institutional incentives to limit the reach of this argument -- the number of people employed in, and the public relations, advertising, and lobbying power of the active management industry far outweigh the resources deployed by companies offering index products (many of which also offer actively managed products). Moreover, active managers are probably more enthusiastic consumers of financial journalism and information products, which would tend, at the margin, to discourage communication of the diversification and indexing message through these channels as well. Again, there is no published research we could find that tested this hypothesis; however it too seems like one well worth examining.

Herding is the tendency for people to discount their own opinions, and instead follow the example set by people they believe to be better informed than themselves. The rush into internet and other technology stocks provides ample evidence that this is a phenomenon that one shouldn't take lightly. If these leaders were seen to be strong advocates of diversification and indexing, it would probably provide a strong inducement for others to follow them in this direction. Still, this begs two questions: who these leaders are, and why they haven't used indexing more extensively (or, perhaps, admitted that they do).

Irrational Causes

The first irrational cause of the behavior we observe may be that investors (and especially the those whose actions are copied by others) are simply overconfident about their ability to pick stocks and/or funds that will "beat the benchmark." Psychological research has shown that overconfidence is widespread (for example, in many areas substantial majorities of the population believe they are better than average). Moreover, these studies have shown that overconfidence is exacerbated by information overload, which clearly characterizes much of the investment management world today.

Our second explanation in this category is more complex, and involves a number of related phenomena that have been well documented by psychological researchers. The first phenomenon is called "prospect theory", which is based on the observation that people's risk appetite changes depending on there current position relative to some reference point. If they are below the reference point, they become risk seekers, in order to make up the lost ground; once they are above it, they become risk averse, to avoid falling below it. If many people saw themselves as being below some reference point, then this could explain their "risk seeking" behavior in choosing low levels of diversification for their portfolios, and an active management approach. What then, could give rise to this widespread feeling of being "below the reference point"?

One part of our theory has to do with the nature of the reference point. Logically, the reference point should be either the compound annual return one needs to earn, or the amount one needs to have saved in order to fund one's liabilities. But, as we all know, in many aspects of our lives logic takes a back seat to emotion. Given this, it is easy to see how at least two different reference points could emerge. The first is the best investment performance achieved by one's friends. As more and more consumer research has shown, people increasingly don't want to be "just average." The disappearance of traditional "mass market" products and stores, and their replacement by "mass luxury" items and outlets is testimony to the power of this trend. Given this, it is easy to see how one could feel frustrated ("below the reference point" in terms of prospect theory) if the performance of one's portfolio lagged behind that of one or more of one's peers (of course it also helps that the only people doing the bragging are usually the ones who beat the index. Those that didn't have a tendency to wander off toward the bar during these conversations…).

This phenomenon is no doubt compounded by something called "hindsight bias", which is a normal human tendency to believe when looking at history that the events that occurred were more likely than previous foresight had estimated them to be. In other words, looking forward, you may estimate that there is a 50% chance that U.S. bonds will be the best performing asset class next year, a 25% chance that it will be U.S. stocks, a 20% chance that it will be foreign stocks, and a 5% chance that it will be foreign bonds. However, if it turns out that in fact it was foreign stocks that performed the best, when asked to recreate from memory your probability estimates, it is almost certain that foreign stocks will be higher than your original 20% estimate. In short, hindsight bias probably adds to our frustration at not having matched our best performing peer's investment results, and so tempts us toward riskier courses of action in the future.

Further adding to the strength of these forces is our tendency to evaluate our portfolio's performance annually, and to do so on an "asset by asset" basis. In a well diversified indexed portfolio, in any given year, some asset classes will have strong performances, and some will have weak performances. This generates the pattern of steady returns that leads over time to truly outstanding long-term performance. However, and this is a big however, in any given year luck alone almost guarantees (in a big enough peer group) that somebody with a less diversified portfolio (perhaps composed of individual stocks or actively managed funds) will outperform our diversified indexer. People simply don't talk about the compound portfolio returns they've earned over the past five or ten years. They talk about what their portfolio earned last year. Actually, it's even more likely that they talk about what the best asset in their portfolio earned last year. And, unless he or she is a zen master, this will naturally cause our good diversified indexer to feel at least a little bit of frustration. And tempt them, perhaps, to go over to the dark side of the force, and either (a) sell out of an asset class whose recent performance has been poor, and/or (b) invest in one whose recent performance has been relatively good (even though there is next to no evidence that past performance predicts future performance). In short, it is easy to see how social interactions can tempt someone to abandon a well-diversified indexed portfolio.

So what can be done to increase the number of people who hold and stick with the well-diversified indexed portfolios that have the highest probability of helping them to achieve their long term financial goals?

I suppose we could start by encouraging more reading of the story of the tortoise and the hare to young investors. Beyond that, other educational efforts designed to encourage diversification and indexing certainly seem to be critical. In particular, they should focus on the importance of having the right goal: not beating the benchmark this year (or your cousin Ralph), but rather adequately funding your own future liabilities. More effort also should be devoted to helping people to understand just how difficult active management actually is, and how few active managers consistently beat the benchmark year after year. Finally, we need to keep in mind that many of the obstacles we face have their roots deep in human nature, and will therefore take years to overcome. Still, considering how far indexing has come over the past thirty years, I'd say we're doing pretty well. We just need to keep it up….