If so, that would be an unfortunate neural response, since all the evidence suggests they really are spouting nonsense.

TO BE NOTED: From The Frontal Cortex:

"Phony Experts

Posted on: March 26, 2009 4:08 PM, by Jonah Lehrer

Nicholas Kristof has a great column today on Philip Tetlock and political experts, who turn out to be astonishingly bad at making accurate predictions:

The expert on experts is Philip Tetlock, a professor at the University of California, Berkeley. His 2005 book, "Expert Political Judgment," is based on two decades of tracking some 82,000 predictions by 284 experts. The experts' forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.

The result? The predictions of experts were, on average, only a tiny bit better than random guesses -- the equivalent of a chimpanzee throwing darts at a board.

"It made virtually no difference whether participants had doctorates, whether they were economists, political scientists, journalists or historians, whether they had policy experience or access to classified information, or whether they had logged many or few years of experience," Mr. Tetlock wrote.

Why are political pundits so often wrong? In my book, I devote a fair amount of ink to Tetlock's epic study. The central error diagnosed by Tetlock was the sin of certainty: pundits were so convinced they were right that they ended up neglecting all those facts suggesting they were wrong. "The dominant danger [for pundits] remains hubris, the vice of closed-mindedness, of dismissing dissonant possibilities too quickly," Tetlock writes. (This is also why the most eminent pundits were the most unreliable. Being famous led to a false sense of confidence.)

Is there a way to fix this mess? Or is cable news bound to be populated by talking heads who perform worse than random chance? As Kristof notes, Tetlock advocates the creation of a "trans-ideological Consumer Reports for punditry," which doesn't strike me as particularly feasible. Instead, I'd argue that the easiest fix is for anchors on news shows to simply spend more time asking pundits about all those predictions that turned out to be wrong. The point isn't to generate a public shaming - although that would certainly be more entertaining than most cable news shows - but to force pundits to genuflect and introspect on why they made a mistake in the first place. Were they too ideological? Were they thinking like a hedgehog? What relevant facts did they ignore? And why did they ignore them?

By asking the right questions, we can make our pundits better. They'll never be perfect, or even worth their inflated paychecks, but perhaps we can help them perform better than chimpanzees throwing darts. Tetlock ends his book with some take-home advice on what we can all learn from the failures of political experts: "We need to cultivate the art of self-overhearing," he says, "to learn how to eavesdrop on the mental conversations we have with ourselves." A big part of that mental conversation is studying the biases and habits that led us to err.

And here, via Steve, is a new study by Gregory Berns of Emory on how even bad "expert" advice can influence decision-making in the brain. Here's the Wired summary:

In the study, Berns' team hooked 24 college students to brain scanners as they contemplated swapping a guaranteed payment for a chance at a higher lottery payout. Sometimes the students made the decision on their own. At other times they received written advice from Charles Noussair, an Emory University economist who advises the U.S. Federal Reserve.

Though the recommendations were delivered under his imprimatur, Noussair himself wouldn't necessarily follow it. The advice was extremely conservative, often urging students to accept tiny guaranteed payouts rather than playing a lottery with great odds and a high payout. But students tended to follow his advice regardless of the situation, especially when it was bad.

When thinking for themselves, students showed activity in their anterior cingulate cortex and dorsolateral prefrontal cortex -- brain regions associated with making decisions and calculating probabilities. When given advice from Noussair, activity in those regions flat lined.

I'm most interested in the reduced activity seen in the ACC, since that brain area is often associated with cognitive conflict. (It's activated, for instance, when people are presented with contradictory facts or dissonant information.) I think one way to understand the influence of experts is that, because they're "experts," we aren't as motivated to think of all the reasons they might be spouting nonsense. If so, that would be an unfortunate neural response, since all the evidence suggests they really are spouting nonsense."

And it turns out that while foxes don’t give great sound-bites, they are far more likely to get things right

TO BE NOTED: From the NY Times:

"Learning How to Think

By NICHOLAS D. KRISTOF

Ever wonder how financial experts could lead the world over the economic cliff?

One explanation is that so-called experts turn out to be, in many situations, a stunningly poor source of expertise. There’s evidence that what matters in making a sound forecast or decision isn’t so much knowledge or experience as good judgment — or, to be more precise, the way a person’s mind works.

More on that in a moment. First, let’s acknowledge that even very smart people allow themselves to be buffaloed by an apparent “expert” on occasion.

The best example of the awe that an “expert” inspires is the “Dr. Fox effect.” It’s named for a pioneering series of psychology experiments in which an actor was paid to give a meaningless presentation to professional educators.

The actor was introduced as “Dr. Myron L. Fox” (no such real person existed) and was described as an eminent authority on the application of mathematics to human behavior. He then delivered a lecture on “mathematical game theory as applied to physician education” — except that by design it had no point and was completely devoid of substance. However, it was warmly delivered and full of jokes and interesting neologisms.

Afterward, those in attendance were given questionnaires and asked to rate “Dr. Fox.” They were mostly impressed. “Excellent presentation, enjoyed listening,” wrote one. Another protested: “Too intellectual a presentation.”

A different study illustrated the genuflection to “experts” another way. It found that a president who goes on television to make a case moves public opinion only negligibly, by less than a percentage point. But experts who are trotted out on television can move public opinion by more than 3 percentage points, because they seem to be reliable or impartial authorities.

But do experts actually get it right themselves?

The expert on experts is Philip Tetlock, a professor at the University of California, Berkeley. His 2005 book, “Expert Political Judgment,” is based on two decades of tracking some 82,000 predictions by 284 experts. The experts’ forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.

The result? The predictions of experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.

“It made virtually no difference whether participants had doctorates, whether they were economists, political scientists, journalists or historians, whether they had policy experience or access to classified information, or whether they had logged many or few years of experience,” Mr. Tetlock wrote.

Indeed, the only consistent predictor was fame — and it was an inverse relationship. The more famous experts did worse than unknown ones. That had to do with a fault in the media. Talent bookers for television shows and reporters tended to call up experts who provided strong, coherent points of view, who saw things in blacks and whites. People who shouted — like, yes, Jim Cramer!

Mr. Tetlock called experts such as these the “hedgehogs,” after a famous distinction by the late Sir Isaiah Berlin (my favorite philosopher) between hedgehogs and foxes. Hedgehogs tend to have a focused worldview, an ideological leaning, strong convictions; foxes are more cautious, more centrist, more likely to adjust their views, more pragmatic, more prone to self-doubt, more inclined to see complexity and nuance. And it turns out that while foxes don’t give great sound-bites, they are far more likely to get things right.

This was the distinction that mattered most among the forecasters, not whether they had expertise. Over all, the foxes did significantly better, both in areas they knew well and in areas they didn’t.

Other studies have confirmed the general sense that expertise is overrated. In one experiment, clinical psychologists did no better than their secretaries in their diagnoses. In another, a white rat in a maze repeatedly beat groups of Yale undergraduates in understanding the optimal way to get food dropped in the maze. The students overanalyzed and saw patterns that didn’t exist, so they were beaten by the rodent.

The marketplace of ideas for now doesn’t clear out bad pundits and bad ideas partly because there’s no accountability. We trumpet our successes and ignore failures — or else attempt to explain that the failure doesn’t count because the situation changed or that we were basically right but the timing was off.

For example, I boast about having warned in 2002 and 2003 that Iraq would be a violent mess after we invaded. But I tend to make excuses for my own incorrect forecast in early 2007 that the troop “surge” would fail.

So what about a system to evaluate us prognosticators? Professor Tetlock suggests that various foundations might try to create a “trans-ideological Consumer Reports for punditry,” monitoring and evaluating the records of various experts and pundits as a public service. I agree: Hold us accountable!"