Asymmetrical Information

How about #6: it's too much trouble to dig in the archives to find out if pundits were right or wrong, and you probably don't remember what they wrote 3 years ago.

Once, in a hotel somewhere, I saw a weather forecaster who compared his predictions with the day before to what actually happened. That takes guts, because most people have probably forgotten by then, and that's only 24 hours.

Rob Lyman is correct - there is no systematic way of determining who is actually more often right.

Davies' fantastically pompous "I was right about every important detail" is, of course (and as I pointed out on the last thread), not accompanied by links to all of his predictions. We cannot evaluate whether he was, in fact, right about anything at all. Or whether he was right about more things than you, Jane.

So there is no way to tell well Davies is a actually a person with a good decisionmaking process or not.

To tell if one pundit is actually more often right than another, so that we could support the "righter" pundit and punish the "wronger" pundit, we need a complete breakdown of their predictions (no cherry-picking, please!). That's not something that anyone ever does.

I vote with #1. We read pundits to get our own ideas/hunches confirmed. I was an Iraq war hawk who like everyone else turned against it. The pundits I enjoy most are those who changed their minds along with me, and for similar reasons. This makes me think that I don't care whether these people are consistent over time. I just care about whether they reflect or echo what I think at any given point.

I think most of those (1, 2, 3 and 6) are true to some extent. I'd add another, and, to forstall accusations that I'm engaging in some of the unpleasantness of other threads, I'll stipulate that this applies to both the left and the right.

People tend, as you say in a subsequent post, to be pretty stubbornly committed to certain viewpoints. When a pundit with whom they agree gets something wrong, they have a tendency to:

(1) Deny that the pundits were in fact wrong, or wrong in a meaningful way. Witness the many commenters on previous threads who still think the Iraqi war was a good idea. They would say the same thing that many of your anti-war commenters are saying "well, they got some of the details wrong, but they were right on the big questions." And yes, if Iraq had turned into a peaceful democracy, I'm sure you would still see some war opponents claiming that the anti-war pundits were right all along.

(2) Claim that the pundits would have been right, but for unforseeable circumstances. "Well EVERYONE thought that Iraq had WMD, and NO ONE realized how incompetant the Bush administration was. The pro-war pundits would have been right but for their understandable lack of knowledge on those points." Again, I use that as a convienient example; I'm sure there are examples going the other way also.

Okay, perhaps my post is just a variation of your reason # 1, but with more detail. :)

I'd say that the answer stems in the greatest proportion from #1 and #2, we don't go to pundits for answers as much as to hear our own thoughts written better than we could have done ourselves. I'd love to see someone's blog have a list of outstanding predictions which they've made, yet still stand by at the top of the page, for everyone to see at any time. I'd add that consumers of punditry probably aren't all that interested in the "rightness" or "wrongness" of their favorite pundits one way or the other, which explains the absence of an easy mechanism for tracking such a thing, as mentioned by the first two commentors. I'd also guess that you're right, that this is at least implicitly known by the pundits, who are in turn not afraid to make even the most outlandish predictions.

"no long, venomous attacks on the character and morals of the hawkish pundits, or their readers."

Sigh. Nice one sided rebuke. Way to calm the waters.

I'll take a blend of (1), (2), and (4) depending on which consumer we're talking about. Personally, I think (3) only happens among the hard partisans who are completely entrenched in (1) to the exclusion of all others, and therefore represent only a small seague; and (5) is unlikely.

However, I think there's a lot more to be said for Rob Lyman's (6): Short attention spans. Larry M's comments, and even some of your own, Jane, are examples of that: We haven't produced a rose garden yet, so obviously everything significant is in the ashcan!

This is a most irritating sort of myopia. It neatly allows those who take the short-sighted position to point to the present and claim all the right answers in support of whatever their present position is. Yet this type of thinking will be rejected by anyone who knows enough history to realize that the correct time reference is years, even a decade or more in some cases. Anyone who cannot muster that kind of perspective, and bring it to bear in a sensible fashion, should abandon foreign policy analysis and return to day trading.

Needless to say, this kind of short-sightedness has a built-in survival mechanism. Even on the ocassions when the naysayers are proven wrong in the long-term, the majority of the pundits -- surprise! -- will adjust their perspective to incorporate the new reality and claim that they held elements of it all along, and the bulk of their judges won't remember enough of the former positions to note that they were really saying something very different, back when it counted.

How about #6: it's too much trouble to dig in the archives to find out if pundits were right or wrong, and you probably don't remember what they wrote 3 years ago.

I think that’s it in a nutshell. I usually have a pretty good memory for things like that but it’s largely because I don’t read a lot of pundits (and generally avoid bloggers who write like pundits) but I would be hard-pressed to remember everything someone wrote even if it was a topic that I was very interested in. Occasionally something comes to mind like when I read a George Will column blasting “big government” Republicans, it triggers memories of an earlier column he wrote telling conservatives to accept that the welfare state isn’t going away and they may as well try to gear it towards “conservative ends.” I would be hard-pressed to find that exact column unless I paid for a Lexis search which I’m not willing to do just to prove a point.

A.S. is correct that it is easy for asshats like Daniel Davies to insist that they were right without of course providing proof of what they had predicted years ago and knowing full well that few people think they’re worth the trouble of trying to research everything they said and when. Why spend time trying to find out who was right about an old argument when there are new ones constantly propping up that require our attention?

Which leaves us kind of where we are now. We have probably the greatest mechanism for researching information literally at our fingertips that isn’t being used for researching old arguments because hardly anyone has the time or interest in doing so.

Another vote for #1.

A.S. keeps saying that Daniel Davies isn't providing any links to his past posts, verifying that his predictions were correct.

February 2003, reacting to a Thomas Friedman column:

"... can anyone, particularly the rather more Bush-friendly recent arrivals to the board, give me one single example of something with the following three characteristics:

"1. It is a policy initiative of the current Bush administration
"2. It was significant enough in scale that I'd have heard of it (at a pinch, that I should have heard of it)
"3. It wasn't in some important way completely fucked up during the execution.

"It's just that I literally can't think what possible evidence Friedman might be going on in his tacit assumption that the introduction of democracy to Iraq (if it is attempted at all) will be executed well rather than badly."

Earlier, ">October 2002:

Well, that'll teach me to use the PREVIEW button.

October 2002:

"A further development in my Iraq policy as well. I am now, after comments from Brad DeLong and others, revising my opinion of the murderousness of the sanctions policy and concluding that it might not be as terrible as a number of quite possibly interested parties have portrayed it. On the other hand, I am also convinced by Max Sawicky's argument that Iraq is likely to be the first excursion of an American policy of empire-building in the Middle East, which is likely to be disastrous under any possible performance metric.

"But, I retain my original belief that improvement in Iraq is politically impossible unless there is some sort of shooting war in the area culminating in the removal of Saddam Hussein. I don't set much score by 'national-building', and don't really believe that what the Gulf needs is more US client states, and I never believed any of the scare stories related to the 'WMD' acronym which is currently doing such sterling duty in picking out weblog authors who don't have a fucking clue what they're talking about. I just think that Saddam needs to go, because it's just one of those Damned Things which Has To Happen. I'm a fatalist, not a moralist.

"So, how can we square these beliefs a) that something has to be done and b) that if something is done, it will be a disastrous imperial adventure by George Bush. Here's how, and it's so simple it's beautiful:

The official policy of D-Squared Digest with respect to Iraq is now that we support a policy of containment until after the 2004 Presidential elections, and after that, we will support immediate war with Iraq if and only if someone other than George W Bush is elected

"I could dress that up by going 'WHEREAS' a lot and turning it into a manifesto, but I've never really had any problems with thinking of new ways to call anyone who disagrees with me an idiot, so it seems a bit pointless to bother. Basically, the idea is that I'll support a war just so long as that idiot currently in charge has nothing whatever to do with it. Thinking about it, I don't want to sign up to a different figurehead for the Perle/Wolfowitz Axis of Idiocy, so maybe I should just tell the truth and shame the devil; I'll only support a war if it's the Democrats fighting it. I would like to find some warblogger with a decent argument against this view; strikes me that I can accept all the arguments about containment, inspection, risk, etc, etc and still hold the statement in bold italics just above. Nobody believes that Saddam will have nukes by 2004...."

To summarize, Davies was correct in predicting that
1. The WMD threat was greatly exaggerated.
2. The execution of the war by the Bush administration would be, in Davies' words, "completely fucked up."

But meanwhile, back at the issue Jane actually raised…

There are a bunch of problems with human decision making, and the gathering of information is only a part of decision making, and typically the part that people who are actually making decisions (as opposed to pundits and second-guessers) get right. People persistently underestimate uncertainty however (see Kahneman and Tversky) and their predictions consistently reflect an over-estimation of the amount they know about a topic, even when they profess to know nothing about it at all. (Note that this is true of both supporters and opponents of the Iraq war.) This phenomenon is, I’m sure, exacerbated by the type of selective memory that you wrote about in an earlier post.

Another problem people have is that the only tool easily available by which to judge the quality of a decision is the outcome, but in the vast majority of cases, and all of the interesting ones, the outcome is subject to a significant amount of randomness. As long as these two facts are true (and the second one is unavoidable) we will spend a lot of our time rewarding people who made bad decisions but got lucky and punishing people who made good decisions but got an unfortunate draw from the deck.

Kierkegaard got it right—“Life can only be understood backwards, but it must be lived forwards.”

This is seriously off-topic flamebait, but really:

I'll only support a war if it's the Democrats fighting it. I would like to find some warblogger with a decent argument against this view;

Because the military accomplishments of the Clinton, Carter, and Johnson administrations are so sterling, and we should have expected Kerry to carry their noble tradition and unmitigated record of success forward.

But on a more serious note, DD does seem to have been more right than many, including me.

The National Review had a bold cover last year that said "We're Winning!" It's very hard to find on their site anymore. But TBogg uses it frequently to rub in their faces. I visit both sites alot. I like the chest thunping and countering that goes on. It's entertaining but rarely informs my opinions.

jl, makes a point that has been on my mind for a very long time. I'm a bit of a baseball and football fan, perticularly so in the effort to use advancede statistical methods of measuring performance and strategy. Once one begins down this path, one becomes acutely aware of how many things people take as proven "facts" are really just the result of random outcomes, and given a larger sample size, the "facts" would change. Thus, giant myths grow about "clutch performances", or players who are "chokers", as a result of extrapolating small sample sizes of performances to make conclusions regarding the quality of a performer.

I think punditry operates the same way. Being right or wrong is so often a random outcome, and the sample sizes are so small (who has ever been given the chance, to use a somewhat unfortunate term, to predict the outcome of 500 wars? Or of 500 decisions to not go to war?), that it really is difficult to tell who was a sage observer, and who was lucky. Obviously, this applies as well to economic punditry, although given enough decades for a econ pundit one could come up with a large enough sample size of predictions to form a useful opinion of the pundit's quality, assuming somebody kept track.

The problem I have had with so much commentary regarding this conflict from the beginning, despite my being guilty of it also, is the faux certainty so many have adopted, no matter where they came from on the political spectrum. I remember cringing when I heard Ken Adelman on cable t.v. describe creating even a semi-decent political culture in Iraq as a "cakewalk". How could anybody have such confidence regarding such a terribly vexing task? Yet, I still supported invasion, and still do, despite my having little confidence in even a minimally pleasant outcome, because of my strong certainty (probable overcertainty) regarding what it would mean to continue the paradigm of paying Islamic despots tribute in return for oil extraction.

What has made me cringe regarding those that opposed the war is their total unwillingness to acknowledge the negative trade-offs entailed in the "no war" decision. This has been particularly galling in regards to those who refer to themselves as proponents of Realism, since it would seem that the first hurdle to becoming a realist would be to acknowledge that every choice of action, or choice of inaction, entails unintended negative consequences. Nope, though, most Realists I've encountered simply assume stasis, and thus embrace the fiction that a status quo can be successfully maintained decade after decade.

Well, I don't have any solution to any of this, other than to say that humility is a quality that many people firmly believe, somewhat unfortunately, is only deficient in those with whom they disagree.

Rob: But on a more serious note, DD does seem to have been more right than many, including me.

Me too.

How about this: the war isn't over. The hawks may yet be right in the longer run - frankly, I'd prefer they are right than the doves be right, since the doves being right would be a _real_ disaster for the world. Only an Islamist or someone afflicted with a severe case of BDS would want to see Cindy Sheehan and her band of commie cohorts claim rhetorical "victory".

I vote for a combination of 1 and 5. To the extent that people read pundits, it is primarily for confirmation of their own positions. Seriously, why else would anyone read Powerline or listen to Chris Matthews or Pat Buchanan? And don't even get me started on Michael Ledeen. The only people who can possibly take them seriously are other pundits who use them as resources to (a) confirm their own biases; or (b) ridiculous counterpoints to their own enlightened positions. Secondarily, punditry is entertainment. We love to watch these know-nothings scream at each other about irrelevancies.

6) People know a stopped clock when they see one.

7) For the love of Christ, Robert Scheer?

Richard Posner wrote a book on this a few years back, the main thrust being that the 'work' of public intellectuals has declined of late and gee, wouldn't it be nice if we had a scorecard for them, because, wow, they're so often spectacularly wrong. People like Lester Thurow, and the gold standard, Paul Ehrlich, dine out their entire lives on having made bold and dunderheaded claims. Which is the point.

Just as the vast majority of people can't find Wyoming or Iraq on a map and have only vague ideas of what terms like 'liberal' anmd 'conservative' mean, while the 2% of us who are interested in this stuff listen to 0.001% of us overinterpret every poll or vote as representing some sea change in the thoughts and minds of the public . . . so too do we overestimate the importance of assclowns like Ehrlich, who are being heard by only a small group of us, and are influencing virtually noone. Leading cheers, yes, but influencing? Naah.

BTW, please read (indeed, subscribe to) Critical Review, esp. the current issue, if you're interested in the issues of public knowledge and the lack of it, and the various factors influencing it (or not).

To answer the question, #2, I guess.

I read pundits to get a different perspective on an issue and see how that point of view affects by belief system, if at all. I like a writer that makes me think. I do not agree with everything Jane says, but she usually gives me something to think about.

For instance, I have a problem with this notion that because there were no WMDs and/or there is a civil war, then the war was wrong. That is hindsight. How could you have known that before the war started? IF we find a mountain of WMDs tomorrow, will that make the war OK?

Look, it's admirable to review your decision making process, but this isn't a math problem. There is no perfect formula to come up with a right answer every time. The US didn't get involved in Rwanda and one million people died. Did we make the right decision? What about Darfur? Is it the right decision to watch tens, maybe hundreds of thousands of innocent people die and do nothing?

This country went to war with itself over 100 years ago and it turned out to be the bloodiest war we have ever been involved in. Was that the right decision?

A.S.: Those doves who made pompous and specific predictions about how, exactly, things would unfold deserve to be called wrong (on those predictions). But this subset of the opponents to the war in Iraq does not constitute the totality of the opposition. Or is it so bleeping inconceivable to you that some of us, while not in possession of dramatic, bold, specific predictions about how the war would unfold, should choose to estimate that the multiple risks made it an unwise adventure? (In that estimation, nearly all doves were right.)

This insidious game of erecting strawmen out of the opposition to the war reminds me of some fundamentalists' occasional ideas about the generality of atheists. Atheists don't believe in God, and therefore they don't have a concept of universal good and evil, etc. (Nevermind that atheists come in all sorts of moral stripes, political convictions, and fundamental beliefs.) In the same vein, all opponents to the war are being painted as fundamentally flawed in their decision-making process, notwithstanding the immense variety of possible reasons (many of them good) for which one could oppose the war, and no matter the fact that those who made outlandish predictions being lambasted by the likes of A.S. and company did not speak for the generality of us.

All of the above contain some truth. In addition, since there is no one keeping a valid scorecard, and no way to slice the predictions into manageable sizes to be testable propositions with finite truth values of 1 or 0, there is no way to tell which pundits have been more right or less wrong.

The Daniel Davies quote above does seem pretty close to the actual occurrence. On the other hand, it doesn't describe the other salient fact in our poor performance in the occupation. That being the intentional acts of our far left in opposing our success in the war, and the intentional acts of our media in eroding our support for the war.

We are like a giant fighting with kittens. We are "losing" only to the degree that we have gotten a few painful scratches, and that we verge on deciding not to win.

Twill00: The Daniel Davies quote above does seem pretty close to the actual occurrence.

Glad to hear it.

On the other hand, it doesn't describe the other salient fact in our poor performance in the occupation. That being the intentional acts of our far left in opposing our success in the war, and the intentional acts of our media in eroding our support for the war.

How salient is this? It was the Bush administration that decided to go in with too few troops, that decided on de-Baathification and the disbanding of the Iraqi army. Mark Danner quotes Bob Woodward, reconstructing a conversation from May 2003:

"If you put this out, you're going to drive between 30,000 and 50,000 Baathists underground before nightfall," Charlie [CIA chief of station] said.... "You will put 50,000 people on the street, underground and mad at Americans." And these 50,000 were the most powerful, well-connected elites from all walks of life.

"I told you," Bremer said, looking at Charlie. "I have my instructions and I have to implement this."

So, Russil,

Can I assume from your comments that since one of mistakes of this war was sending in too few troops in the beginning that you are in favor of sending in more troops now, like the president wants to?

Or, is this a case of Bushed if you do, Bushed if you don't?

Reagan Fan: Can I assume from your comments that since one of mistakes of this war was sending in too few troops in the beginning that you are in favor of sending in more troops now, like the president wants to?

No. At this point it'd be too little, too late. I think Bush ought to have listened to the Iraq Study Group: make continued US military and economic support for the Iraqi government conditional on progress.

Russel Wvong, are you sure that wasn't a conversation Bob Woodward held with Bob Casey after his death? Woodward does seem to have an amazing ability to get people to say rather interesting things...

Jane, in #6 you misspelled "cone", for some reason or other. Or at least, I wish you had...

ellipsis: good point. But here's Jay Garner (who was there) saying the same thing on Frontline.

Because there is no real incentive to vote, to the extent that punditry is about voting, there is no incentive to find the right pundit. Granted, there's much punditry on financial issues, such as Argentina's default, and that's got plenty of incentives. A lot of very smart people were wrong and lost money on Argentina. I think such mistakes are why the old hiarchical model of info is getting eaten alive by distributed forms, like this nice blog. The key is figuring out how to average your so-so advice, Jane, with everyone elses. Cheers.

Invading Iraq was the right decision then and it still looks the right decision now. The benefits from the decision far outway the costs.

I don't know anyone whose predictions were proved accurate. Not one.

Hell, we can't even get a news media to accurately report what has just happened. If reporters can't even tell us what happened, why should we expect anyone to accurately predict the future.

And how would we even judge the pundits' performance if we can't even get accurate info on the past?

To answer Jane's question, I think it's # 1. People want to see their views confirmed, and they read columnists and watch TV pundits accordingly. However, and here is where I differ from Jane, I don't think there's a free market in ideas. If you're interested in public policy, but not to the extent of reading blogs, and if you want to read something written above the junior high school level, your choices are constricted because of media concentration. If you're a conservative in New York City or Boston or a liberal in Los Angeles or Washington, D. C., you're pretty much out of luck.

The original post sounds about right.

Comments are Closed.