Asymmetrical Information

From the desk of Jane Galt:

Why pundits are like businessmen

Louis Menand has an article in the New Yorker which ought to make me, a professional news analyst, despair:

Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study. Knowing a little might make someone a more reliable forecaster, but Tetlock found that knowing a lot can actually make a person less reliable. "We reach the point of diminishing marginal predictive returns for knowledge disconcertingly quickly," he reports. In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals--distinguished political scientists, area study specialists, economists, and so on--are any better than journalists or attentive readers of the New York Times in reading emerging situations. And the more famous the forecaster the more overblown the forecasts. "Experts in demand," Tetlock says, "were more overconfident than their colleagues who eked out existences far from the limelight."...

The experts' trouble in Tetlock's study is exactly the trouble that all human beings have: we fall in love with our hunches, and we really, really hate to be wrong....

The expert-prediction game is not much different. When television pundits make predictions, the more ingenious their forecasts the greater their cachet. An arresting new prediction means that the expert has discovered a set of interlocking causes that no one else has spotted, and that could lead to an outcome that the conventional wisdom is ignoring. On shows like The McLaughlin Group, these experts never lose their reputations, or their jobs, because long shots are their business. More serious commentators differ from the pundits only in the degree of showmanship. These serious experts, the think tankers and area-studies professors, are not entirely out to entertain, but they are a little out to entertain, and both their status as experts and their appeal as performers require them to predict futures that are not obvious to the viewer. The producer of the show does not want you and me to sit there listening to an expert and thinking, I could have said that. The expert also suffers from knowing too much: the more facts an expert has, the more information is available to be enlisted in support of his or her pet theories, and the more chains of causation he or she can find beguiling. This helps explain why specialists fail to outguess non-specialists. The odds tend to be with the obvious.

The problem as I see it is that the market for punditry has skewed incentives. There is no reward for being boring and right, nor any punishment for being novel and wrong. But there are big rewards, in the form of book contracts and lecture fees, for being novel and right. Pundits are thus tempted to act like executives with fat option packages. Executives with stock options get paid only if the stock appreciates considerably, putting their options "in the money"; but unlike investors, there is no downside for them. They are just as badly off if the stock stays at its current level, keeping their options "out of the money", as they are if the company implodes and the price of the stock drops to 50 cents a share. They are thus tempted to take risky steps which have high upside potential. Similarly, the expected value of an obviously right prediction is exactly the same to the pundit as that of a spectacularly wrong one. They are thus tempted to make risky predictions, hoping that one hits and they strike it rich.

I have myself observed the tendency of journalists to believe that their spectacularly wrong predictions (i.e. my near certainty that Argentina would never default on hte IMF) were merely bad luck, while their correct predictions are evidence of unusual intellectual acumen. Thus, in addition to their likeness to executives with options packages, pundits also closely resemble mutual fund managers, and the people who invest with them.

Statistically, mutual fund returns show little evidence that their managers are particularly sharp; a manager who lead the league last year is no more likely than any other manager to repeat the phenomenon this year. But managers rarely attribute their good years to being in a lucky spot on the bell curve; they see them as the natural result of the wise decisions they made throughout the year. And because there are so many mutual funds out there, some managers by pure chance manage to put together strings of good years, just as others, by almost equally random chance* will have a run of bad years. If you flip a coin a thousand times you will, purely by chance, get some strings of heads and tails that seem non-random to the casual observer, who may be tempted to attribute the strings to the coin-flipper's unusual skill. This is why investors, stupidly, pile into mutual funds that have had a couple good years behind them. They tend to get the same results that you could expect if you decided to put money down on your "hotshot" coin flipper continuing his winning streak.

In other words, a bet on the predictions of famous pundits is a sucker bet. As Daniel Drezner suggests, political bloggers everywhere are undoubtedly congratulating themselves on having been right all along.

*while there's no evidence that good mutual fund managers can beat the market, there are bad managers who consistently underperform it. Luckily for us, they are quickly fired and their funds closed.

Posted by Jane Galt at November 30, 2005 8:28 AM | TrackBack | Technorati inbound links