Unquestioned Results and Academic Groupthink

JS Mill
John Stuart Mill

As someone venturing into academia, particularly into academic research about public policy (policy, not politics), I’ve had to come face to face with some brief facts about the environment I’m about to enter.

  1. Self-selection into academia (and especially social science) tends to attract politically liberal people (more on that later).
  2. Underlying worldview and values play a major role in what research questions are asked and how they are asked.
  3. When results arise, the prevailing intuition of reviewers and discussants may create a false sense of consensus.

Case in point: a paper that was widely circulated and reported originally found that political conservatives exhibited personality traits linked to authoritarianism, less social desirability, and psychoticism. This analysis wasn’t even a central finding of the paper, but it got a lot of attention.

It turns out the opposite was true, and the authors simply didn’t realize their political scale was backward.

Not only did it breeze past reviewers, it also took a number of years for–wait for it– a graduate student to identify the errors because the results were so at odds with the literature.

What is sad is that the scale wasn’t the only issue plaguing the analysis.

As Andrew Gelman told Retraction Watch:

“I don’t find this paper at all convincing, indeed I’m surprised it was accepted for publication by a leading political science journal.  The causal analysis doesn’t make any sense to me, and some of the things they do are just bizarre, like declaring that correlations are ‘large enough for further consideration’ if they are more than 0.2 for both sexes.  Where does that come from?  The whole thing is a mess.”

What allowed for such a “mess” to go unnoticed? Assuming no ill will on the part of the researchers, I can only say that the intermediate findings may have confirmed an understanding common to the ideological identity of the prevalent group, leaving very few to deeply question the results. After all, an academic observation that could be construed as insulting (I’m sure someone, somewhere would think so) injures no one if so few (or no one) in the room identifies with the target group; it’s hard to think there’s a problem. No one has a reason to raise an eyebrow.

Such groupthink appears to be a core reason for Jonathan Haidt, et al’s elevated criticism of American academia having made such a strong leap leftward over the past two decades.

As they quote John Stuart Mill:

“He who knows only his own side of the case knows little of that. His reasons may be good, and no one may have been able to refute them. But if he is equally unable to refute the reasons on the opposite side, if he does not so much as know what they are, he has no ground for preferring either opinion… Nor is it enough that he should hear the opinions of adversaries from his own teachers, presented as they state them, and accompanied by what they offer as refutations. He must be able to hear them from persons who actually believe them…he must know them in their most plausible and persuasive form.”

Advertisements

6 Replies to “Unquestioned Results and Academic Groupthink”

  1. Can you please try and make sure you say “Leftist” rather than “Liberal” in these situations? This is something a lot of us are trying to raise consciousness about. Jon Haidt has tried to make this point too.

    Liberal really needs to remain in the sense that Mill meant it. It’s about openness. Being open to hearing opposing views. When you define it like that, you instantly see that there are plenty of conservative liberals. And plenty of progressives who are not remotely liberal.

    Like

  2. Having abandoned academia to do something useful (I’m a cabinetmaker in a high end manufactory), I suspect that your quote of JSM is dead on. Most academics who write the drivel in the paper you quote,

    “…the intermediate findings may have confirmed an understanding common to the ideological identity of the prevalent group, leaving very few to deeply question the results. After all, an academic observation that could be construed as insulting (I’m sure someone, somewhere would think so) injures no one if so few (or no one) in the room identifies with the target group…”

    is exactly why this crap was allowed to stand for so long. When you believe that your opponents are evil, as most Liberals/Progressives do, then when you come up with “results” that confirm your world view, you won’t question them, because you CAN’T question them without shattering said world view…

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s