Fundamental flaws in (cancer) research

Watching a TED talk by Ben Goldacre recently, my attention was drawn to an excellent Nature article on fundamental flaws in cancer research. The Comment Raise standards for preclinical cancer research (subscription required), by Glenn Begley and Lee Ellis, discusses some systematic weaknesses in basic biomedical research and proposes some solutions that would resolve some of these problems.

Nature 483:531–533 (29 March 2012) doi:10.1038/483531a

Nature 483:531–533 (29 March 2012) doi:10.1038/483531a

As part of their work at the Amgen pharmaceutical company, the authors have tried to replicate the findings in 53 “landmark” papers reported to reveal important advances in understanding about the molecular biology of cancer. Despite their best efforts, including contacting the scientists responsible for the original studies, getting resources from them and, in some cases, visiting their labs to repeat the protocol there, Begley and Ellis only managed to successfully reproduce the published results in 6 (11%) of cases. We are not told which experiments were replicable, or perhaps more importantly which were not, since confidentiality agreements had been made with several of the original authors (a point that was made post hoc in a clarification statement). Continue reading

Finding research articles that are worth finding

An enormous amount of scientific literature is generated each month

The world is awash with scientific papers. Even if we restricted a survey to research within biological sciences, I guestimate that there are more papers published each month than your average academic could be expected to read in a lifetime. In these rich fields of information, how are students unfamiliar with the genre  to develop the ability to discern the wheat from the chaff (let alone the weeds)?

At the University of Leicester, we have a task for undergraduates, conducted towards the end of their second year, in which they produce a poster describing a particular research method (selected from a short list relevant to their chosen discipline). As part of the exercise, the students need to choose a primary research article which illustrates one application of that method.

It is fair to say that many of the students initially struggle to select an appropriate paper*. There are several reoccurring  problems:

  • Failure to distinguish between a research article and a review
  • Failure to recognise that all journals are not equal in terms of their academic quality and rigour of the work they publish
  • Selection of papers that do not really utilise the technique that should be the focus of their poster.

In truth, the ability to select the right kind of paper is one of the important learning outcomes from this exercise; these students will commence their final year dissertations immediately after they return from summer vacation and need to avoid wasting hours reading papers that are not worthy of their attention. However, since I frequently find myself making the same points in email correspondence with individual students, I felt it was worth using this forum to share some of my overarching reflections on the fine art of finding appropriate research articles.

Research Article or Review?

Most searches these days are conducted online

When we’ve been involved in academia for many years, knowing whether a paper is a primary research paper or a review comes as second nature. This is not necessarily true for inexperienced students. In their defence, the variation of the naming of research articles in different journals does not help. Cell and EMBO Journal call them “Articles”. Nature also has “Articles” but the majority of original research is labelled as “Letters” and in Science primary literature is mostly “Reports” .

Although it ought to be possible to spot a research article by the sub-sections it contains (i.e. Abstract, Introduction, Materials and Methods, etc…) these are not necessarily given the same names in all journals, and the order in which they included can be different. Of course, if the journal has the word “Review”, “Trends in…” or “Current opinions in…” then it is likely to include, almost exclusively, articles summarising the research of other scientists rather than containing original reports of new experiments . But in other journals a review might actually be called “Perspectives”, “Commentary” or “News and Views” to name but three.

Review articles as a mean to an end

For the specific activity we set our students (finding a primary research paper that demonstrates the use of a particular technique), reviews are not going to be a suitable destination. This does not mean, however, that review articles are of no merit in the search for good quality experimental data. In some senses, the authors of a review article have done crucial legwork for you. Already experts in the field, they have read broadly about the topic and will then have selected what they consider to be the most significant recent experiments for their reflections. Looking at the reference list in a review can therefore be an efficient way to shine a light on the best primary literature.

Is it a good paper?

If you are not familiar with a research technique or particular discipline, how can you know whether a paper is a “good” paper? There are three useful clues. The first is the one we have just described, namely do authors of reviews rate the work as worthy of their attention?

Secondly, where has it been published? Like it or not, there is a hierarchy of journals; some titles are in the Premier League, some are in the Championship, and some are non-league. Experienced heads know that research published in Nature, Science, Cell, Proceedings of the National Academy of Science and EMBO Journal is considered (usually with some justification) to be more worthy than papers coming out in the Journal of Knitting and Spectroscopy, but recognising this is, once again, an aspect of the maturing of undergraduates into fully-fledged scientists.

Thirdly, how many times has the work been cited by other researchers? If work has been in circulation for fifteen years but has only been quoted twice during that period, and both times by the author of the original paper, then it is fair to conclude that it did not contain ground-breaking discoveries. One of the useful features of Web of Knowledge is the citation count. If you don’t have access to Web of Knowledge, Google Scholar has a similar (though slightly less rigorous) feature.

Finally, if you want to refine a search so that it is restricted to only those “Premier league” journals name above,  then why not go to a journal’s own search engine rather than, or in addition to, a more generic search tool? (e.g. for Nature, for Science).

* Note, if you are wondering how this squares with my recent description of a  source evaluation exercise for first years, I need to point out that this is NOT the same cohort.

Casting new light on the origins of Plasmodium falciparum

Anopheles mosquitos spread malaria

Malaria is a disease that has killed more humans than any other during our history; it still infects some 500 million people each year, killing about 2 million. Although a bite from a female Anopheles mosquito is central to the transmission of malaria, it is in fact a Plasmodium pathogen harboured within the mosquito that causes the disease. There are five distinct species of Plasmodium known to cause malaria – of these P. falciparum is the most significant both in terms of quantity and severity of infection.

Having entered the body via a bite, the pathogen migrates in the bloodstream to the liver. From here it will re-enter the blood stream and colonise red blood cells. When sufficient multiplication of the pathogen has occurred, the blood cells will burst, releasing more pathogens into the blood. The bursting occurs on a 48-72 hour cycle and is accompanied by peaks in fever and other symptoms at these times.

Persistent and repeated damage to the red blood cells leads to a shortage of erthyocytes and associated symptoms of anaemia. Rarely, P. falciparum can cause blockage of blood vessels supplying the brain, resulting in insufficient supply of oxygen, leading to brain damage, seizure and/or coma (all data taken from the NHS Choices website).

New evidence undermines traditional view on the origins of P. falciparum

In a September 2010 paper in the journal Nature, Beatrice Hahn and colleagues investigated the evolutionary origin of P. falciparum (Liu et al; doi:10.1038/nature09442). Prior to publication of this research, it was believed that P. falciparum infecting humans had evolved from P. reichenowi in chimpanzees. Until recently this had also been considered as a classic case of “host-parasite co-divergence”, i.e. that when the human line had diverged from the chimpanzee line – between five and seven million years ago – they were both already infected with the same ancestral Plasmodium species which had subsequently evolved apart at the same time that the hosts had evolved. However, three studies already published in the previous 18 months had questioned this chronology (whilst still believing that chimpanzees were the reservoir from which the ancestral version of P. falciparum had passed to humans). Continue reading

A new model for interaction between science research and TV?

A fascinating thing occurred this week. The website of top-notch scientific journal Nature uploaded the preprint of a paper on research looking into the alleged benefits of brain training games.

In and of itself this news may not sound revolutionary; Nature frequently publishes articles on neuroscience (and, I suspect, will be doing so more and more in coming years).   The thing I find interesting about this particular example is the fact that the research was initiated by the BBC’s peak-time science programme Bang Goes The Theory (awarded an honourable mention in last year’s round-up of Science TV). So what we have is television investing in science conducted by a recognised leader in the field of brain research (Adrian Owen, as also seen here) with the net result being a paper in a leading journal as well as an interesting programme.

Now clearly there is a lot of fundamental and important science that needs doing but will never attract the gaze or the funding of the BBC, Discovery Channel or so on. Nevertheless is this serves as a paradigm for a relationship that generates cash for research and at the same time enhances the quality and integrity of the science being discussed on the TV, that’s got to be a good thing. Right?