Add something “sciencey” to improve your plausability

There are many reasons why I am grateful to have spent some of my summer reading Ben Goldacre’s excellent book Bad Science, including the fact that it brought to my attention a paper The Seductive Allure of Neuroscience Explanations. The article is an account of experiments conducted by Deena Weisberg and colleagues at Yale University, and was published in the Journal of Cognitive Neuroscience in 2008.

jcn2Recognising that neuroscience is an area of research that fascinates the public and where discoveries are frequently picked up by the general press, Weisberg et al generated four explanatory statements for each of 18 different psychological phenoma. In each case the four statements represented:

  • a good explanation without specific mention of neuroscience
  • the same good explanation with the addition of plausible, but logically irrelevant, neuroscientific details
  • a bad explanation without specific mention of neuroscience
  • the same bad explanation with the addition of the same plausible (but irrelevant) neuroscience as in the second example

Satisfaction (on a 7-point scale) engendered by the various statements was tested with three cohorts:

  • naive adults (n=81), i.e. individuals with no formal neuroscience training
  • students from an introductory cognitive neuroscience course (n=22)
  • experts in neuroscience (n=48), although the definition of “expert” in this context was quite generous – 6 members of this group had completed an undergraduate course but were yet to start their advanced degrees, 29 were currently in graduate school and the other 13 were beyond grad school.

In developing their neuroscience explanations, the authors held to three important criteria: (1) the phrasing would indicate that this was a field in which knowledge was already established, (2) the same information would be added to both the good and the bad explanation, and (3) the neuroscience information should not alter the underlying logic of the explanation such as it was before addition of the extra ‘science’.

With all three cohorts, the participants were informed that the study being reported was scientifically robust, but that the explanation offered may not be genuine. For both the ‘naive’ and ‘expert’ groups any individual always saw only statements that contained neuroscientific information (but might be a good or bad explanation), or they never had the additional details. This approach was adopted so that subjects were not alerted to the fact that some of the explanations were fuller than others. Unfortunately the relatively small size of the current undergraduate cohort meant that the method was different, any individual could be exposed to all 4 different types of explanation. This is a shame, since it does require some caution when interpreting side-by-side comparisons of the three cohorts.

So what were the findings? For all three groups, participants rated the good explanations as more satisfactory than the bad ones. Both the naive group and the students found explanations with added neuroscience to be better, and the effect was more striking for the bad explanations than for the good. With the experts, however, addition of the spurious scientific details to the good explanation actually led to a reduction in their satisfaction – an indication that with their fuller knowledge they saw through the vacuous additions.

The authors argue that the findings with novices and students may be manifestations of the “seductive details effect”. Previous studies have suggested interesting-but-irrelevant information can have a detrimental effect on cognitive tasks, e.g. memory tests.

They also point to a general observation that individuals respond more favourably when given a reason for a request, however obvious. They cite one example of such “placebic” information in which research subjects were more amenable to letting someone carry out some photocopying if they added the phrase “I have some copies to make” to a bland request “May I use your Xerox machine?”

In Bad Science, Ben Goldacre argues that ‘quacks’ exploit this phenomenon by dressing up their claims with sciencey-sounding explanations. This reminded me too of the BBC documentary Professor Regan’s Medicine Cabinet (first shown in April 2009) in which, amongst other things, Regan dresses up a test of pills for treating insomnia with all sorts of detailed instructions about what must, and what must not, be done in order for the therapy to be effective. The participants stuck to the rules and reported beneficial effects of the pills, despite the fact that they were – in fact – sugar cake decorations.

Weinberg D.S., Keil F.C., Goodstein J., Rawson E. and Gray J.R. (2008) The seductive allure of neuroscience explanations Journal of Cognitive Neuroscience 20:470-477

Advertisements

Leave a comment

No comments yet.

Comments RSS TrackBack Identifier URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s