Somewhat belatedly I have been catching up on a couple of reports about the future of Science teaching in Europe. Both were prompted by widespread concern that school science in its present form is not meeting the needs of society for the 21st Century. The decline in students’ attitudes towards science – apparently universal across Europe – is a particular worry.

Science Education Now: A renewed pedagogy for the future of Europe was published in 2007 and Science Education in Europe: Critical reflections in 2008

Published in 2007, Science Education Now: A renewed pedagogy for the future of Europe was written at the behest of the European Commission with the specific objective “to examine a cross-section of on-going initiatives and to draw from them elements of know-how and good practive that could bring about a radical change in young people’s interest in science” (p2).

The second paper, Science Education in Europe: Critical reflections follows on from two seminars held in 2006 at the Nuffield Foundation in London. The final report was published in January 2008.

It is not my intention to analyse the documents in detail; both are written in a clear and accessible style, and at only approximately 30 pages each they are eminently worthy of your direct consideration. Here instead I will reflect on a couple of the major issues that struck me as I read the reports.

.

What is the purpose of science education?

A fundamental problem relating to science education is apparent from the outset; namely the objective of the whole process. There has long been a tension between ’school science as foundational knowledge for future scientists’ and ’school science as a mean of equipping the general citizen for engagement with science as they will encounter it in everyday life’.

In most countries science is now compulsory and as such, it is argued, the emphasis must be on the second of the two aims – i.e. promoting scientific literacy for all. Osborne and Dillon, authors of the Nuffield report, see this as warranted on both moral and economic grounds – it is poor management of resources to flog the majority for a programme geared for the minority who will take science to a higher level.

This was the motivation of the major GCSE curriculum changes that took place in the UK from October 2006, with much more emphasis placed on the understanding the process of science  - “How Science Works” – than on fact regurgitation. Advocates of the changes point out that the new curriculum benefits both the general student and the future scientist, though the latter will need additional instruction to establish the core knowledge necessary for higher study.

Critics have suggested that the baby has been jettisoned with the bathwater, that the new approach is “more suitable to the pub than the classroom“. I suspect that many of my science colleagues would intuitively feel nearer this latter position. Of course no-one is yet able to measure the effect on future scientists; the first students that took the new GCSE specifications will not start undergraduate courses until September 2010 at the earliest. I have genuine expectations that we will encounter a new breed of more science-savvy students from the autumn.

Of course one of the perennial dilemmas in education in general,but particularly science education, is whether we will be afforded time to see the current developments through before some new government initiative will force additional changes to be made. Like incoming Students’ Union sabbatical officers, new political leaders have an overwhelming desire to do something differently and thereby leave their legacy, but it does make it difficult to tell which innovations were genuinely beneficial when nothing is left to run its course.

.

Induction v Deduction, Inquiry v Instruction

There are some nice metaphors in the reports; I particularly like the analogy given by Osborne and Dillon that much traditional science teaching is like “being on a train with blacked-out windows – you know you are going somewhere but only the train driver knows where” (p8). Both documents hold that this type of deductive, “top-down” transmission delivers fragments of knowledge whose relevance does not become clear until you have got to your destination; if you fall by the wayside before reaching that point then it will never make any sense. Apparently random snippets of information hold no appeal for all except quiz addicts, and thus the majority, especially girls, switch off.

Instead, inquiry-based science education (IBSE) is advocated, a model in which a structured problem, ideally with a hands-on or practical dimension, comes first and then students are encouraged to develop their critical thinking and reflection skills whilst formulating an understanding of what is going on.

This latter approach has much going for it, but there are significant difficulties in adopting pedagogy of this kind. One problem is the fact it is fundamentally different to traditional approaches – you cannot get to it by minor tweaks of current practice. Secondly, ‘correct’ scientific interpretations do not always flow intuitively from observation. Thirdly, if IBSE is to avoid descent into “the blind leading the blind” it requires the science teacher to feel confident in their own underlying knowledge of the subject and in their classroom management skills, such that they will allow free-flowing discussion to occur before pulling together the salient points at the end.

This is a major sticking point. Combined with research that shows attitudes towards science are frequently cast in stone before the age of 14, a pivotal role is played by the primary school teacher. We know that relatively few primary school teachers have science backgrounds and many feel uncomfortable with even straightforward scientific experiments. There is thus a vital role to be played in equipping primary school teachers for this task. Good work is being done – both papers put a spotlight on the POLLEN programme, an initiative in 12 European countries trialling continuing professional development for primary school teachers in a way that will be scaleable across the wider community.

.

Careers from Science > Careers in Science

The final point that struck a chord with me was the need to convey to students that studying science can lead on to more varied (and more interesting) careers than simply wearing a white coat and working in a lab. Osborne and Dillon talk about an emphasis on Careers from science not just Careers in science. This is one of the things I am trying to convey to undergraduate scientists through the Careers After Biological Science seminar programme where we try to include a blend of ‘obvious’ and less obvious careers that can follow advance study of science (see our sister site biosciencecareers for more details). A rung or two further down the educational ladder I take it that this is also the goal of the scienceandmaths.net programme currently being advertised in cinema trailers before the feature film.

Development of appropriate curricula, appropriate pedagogy, appropriate assessement tasks and appropriately-trained teachers are going to be crucial in producing scientifically-literate societies and socially-literate scientists in the 21st Century.

Harper Perennial edition (2009)

If you have not yet read Ben Goldacre’s book Bad Science, then I thoroughly recommend that you do. As readers of his regular Guardian column or his website will already know, Goldacre has embarked on a campaign to root out example of pseudoscience and shoddy science whereever they may be found.

All the usual villians are present – homeopaths, nutritionists, slack journalists, pharmaceutical companies and AIDS dissenters. Some are mentioned by name, but given their alleged predilection for litigation, and since I do not have the time, the money or the inclination to do battle with them in the courts, I shall not repeat their identities here!

It would be wrong, however, to give the impression that Goldacre is merely on a crusade against high profile exponents of “bad science”. True, the author does sometimes betray a little too much glee as he places a bomb under the throne of a media ”health expert” (in a way that I found disturbingly reminiscent of the Physiology lecturer, when I was a first year undergraduate, recalling his boyhood experiments on frogs). Nevertheless, Goldacre is keen to emphasise that his purpose is to “teach good science by examining the bad” (p165 in my copy), adding that ”the aim of this book is that you should be future-proofed against new variants of bullshit” (p87).

It seems to me that Goldacre is correct in his assertion that the public needs help in ‘bullshit-spotting’ and that this book is an extremely valuable tool in achieving that goal. Scientific colleagues will (hopefully!) be familiar with at least some of the pitfalls of poor study design, inappropriate use of statistics and outright spin that lead to dramatic-but-spurious headlines in the newspapers. I am, however, convinced that there is plenty here that will improve the scientific literacy of undergraduates in medicine and bioscience subjects, as well as a more general readership.

Trial design
For an experiment involving human subjects to have at least some hope of generating objective data, it is important that the research method includes:

• control groups – you need something against which to compare your  intervention, whether it be a placebo or sham treatment, or the best treatment currently in use;
• appropriate blinding - i.e. that neither researcher nor participants know during the trial which individual is receiving each intervention;
• randomisation - trial subjects need to be assigned to different regimes in a genuinely unbiased way (some randomisation protocols are actually open to significant abuse, albeit subconscious);
• documentation - when the work is published, the account needs to include suitably transparent and complete details of the methods and the results such that any reader will know how the study was conducted and can therefore have a sporting chance of spotting the glitches.

The case of the widely-reported Durham trial of fish oil tablets containing Omega-3 fats (Chapter 8, ‘Pill solves complex social problem‘) is a chastening tale of ways in which poor research methodology can effectively ruin a study before it has even started. Alarm bells ought to have been triggered as soon as the trial (I will call it that for simplicity, although those involved in the research have shyed away from this term) was trumpeted in advance as a test to prove the effectiveness of fish oils in boosting academic performance. The fact that the participants knew that they were in a trial has been shown in itself to ellicit improvements (the so-called ‘Hawthorne effect’), even without the media scrum that accompanied this particular trial. Add to this the influence of potential ‘confounding factors’ (see below) and this study was never going to give clear and unequivocal results.

Common mistakes involving science literature
Goldacre’s critique of ‘nutritionists’ highlights four frequent errors in the way that science literature is handled. These are:

• extrapolating and overinterpreting data – For example studies conducted on isolated cells in vitro, can provide useful pointers for future studies in humans, but it is wrong to naively take findings from cell-based work and assume the equivalent is true in vivo in a whole organism. To purloin one of Goldacre’s favourite phrases “I think you’ll find it’s a bit more complicated than that” (p100)
• extrapolating from observational studies to make claims that require an interventional study to be conducted – ‘confounding variables’, that is differences between individuals that may or may not be linked to the factor under investigation, are hard enough to control in a study where the researcher is deliberately intervening in the partcipants’ lives to measure any apparent effects. If the study is merely observing differences between people reported to have an important lifestyle or dietary factor, there may be a lot more going on. Superficial analyses are prone to come up with erroneous conclusions.
• Cherry-picking only results that fit the hypothesis – it is, as Goldacre points out, a facet of human nature both to see patterns in data and to be more receptive to results that fit your expectation than those that do not. We need therefore to guard against selectively quoting only experiments that give the results that we want, and ignoring data (possibly the majority of findings) that don’t fit our model. This is why ‘systematic review‘ of all of the data on a particular topic is an essential process.
• Referring to studies that are not published in peer-reviewed journals, and frequently not published at all – it is bad enough when conference papers and press releases are reported with the same gravitas and authority as experiments which have been scrutinised by experts in the same field as part of the peer-review process. It is even worse, however, when some interviewees are prone to make specific claims such as “a study published just last week in America has described the same effect we see here” whilst it later turns out that no such article exists. In written work, some authors have increasingly given their books a spurious air of authority by adopting the trappings of good citation practice, e.g. use of superscript numbers to direct readers to their sources. When you flick on to check the reference, however, it turns out to be a non-scholarly document or something that they themselves have said on a different occasion.

Lies, damned lies and statistics
Statistics are clearly vital in substantiating the findings of any kind of trial and Goldacre attacks abuse of statistics on two fronts. Firstly, there is the deliberate use of an inappropriate statistical test to generate a positive-sounding number. Pharmaceutical companies are said to be guilty of this sleight of hand, and it requires a certain amount of statistical nous in order to detect when this crime is being perpetrated.

Secondly, there is the way that the numbers are presented to the public. Newspapers are prone to report the ‘relative risk increase‘, i.e. the percentage increase in condition X when presented with risk Y because it generates the most attention-grabbing numbers. The shock statistic “reading science-related blogs increases the chance that you’ll have a heart attack by 50%” may alarm you (so just in case, let me say straight way that I made this up).  A very different impression is given if we consider the ‘absolute risk increase‘ which state that “reading science-related blogs increases the chance that you’ll have a heart attack by 0.2%”. Goldacre recommends that there ought to be a move towards quoting ‘natural frequencies‘, i.e. as intelligible numbers. In this case, therefore we might say “reading science-related blogs increases the chance that you’ll have a heart attack from 4 in every 1000 people if you don’t, to 6 in every 1000 people if you do”.

Putting Bad Science to use in formal education
Are there ways in which Bad Science might be employed as a teaching tool in either secondary or tertiary education? The specifications for GCSE Science in England and Wales were altered in 2006 to place greater emphasis on “How Science Works“, and A levels were similarly altered in 2008 when this cohort passed on to the higher qualification. The reading level required to appreciate Bad Science probably procludes recommending it for the majority of 16 year olds. I believe, however, that the text would make an excellent resource for students of A level biology and/or General Studies. I do not know if the publishers have considered producing a structured guide based on the book or inclusion of end of chapter  study questions in future editions, but there is certainly scope for this.

Similarly, the book would be valuable reading for first year undergraduates in Medicine, Bioscience or Journalism. I think there would be more merit in having this as prescribed reading for a Year One skills or introductory module than several of the more ‘academic’ alternatives.

As an admissions tutor, I receive several e-mails each summer from students starting the following term and asking which textbooks to buy. My consistent response this time around has been to recommended that they read Bad Science now and wait until the course has started before they part with money for a chunky Biochemistry text.

Gripes
This is not to say that Bad Science is without faults. I do have a number of minor quibbles with the book, but I would say for the most part the fault lies with the editorial process rather than with the author per se.

Haven’t I read that before? Understandably much of the content of the book has already seen the light of day in shorter pieces in the Guardian’s Bad Science column. Repetition and/or poor ordering (by which I mean a point is introduced at length after it has already been previously noted) betray the ‘cut and shut’ nature of some of the present material. As an example of the former, we are told twice in consecutive paragraphs on page 113 about the crusade led by cereal magnate John Harvey Kellogg against a particular personal vice. Similarly, the fact that Durham council altered a press release on their website sometime after its release in order to remove the word ‘trial’ is mentioned on pages 143 and 149.

Examples of the ‘introduction after being stated’ phenomenon include the mention on page 157 that Equazen had been acquired by Galenica, follwed on p160 by a fuller account of this transaction in a tone that gave the impression it was ‘new news’. Similarly we are told on page 313 that some researchers did “something called a ‘case-control’ study” despite the fact that case-control studies were amongst the variety of experimental models discussed on page 103 and pages 295-296.

Page numbering: The cover of my edition of the book (Harper Perennial, 2009) trumpets the addition of an extra chapter. This material has not been added at the end of the text, but rather inserted at the appropriate point in the unfolding ’story’. In consequence, page numbering downstream of the insertion is altered. Although this has been recognised in the index, there are several examples of in-text cross-references where the page numbers are now 17 out. (In case anyone with influence on the next version is reading this review the reference on page 106 to p240 should be p257; page 282 should cite p294 not p277; page 330 should point to p293 not p276).

Referencing: Bad Science is intended to be a popular book not an academic tome. As such, it would be completely inappropriate for the text to be peppered with citations in a way that would interrupt the flow. I think the solution chosen here works very well – the notes in the back use page numbers and a short quote from the text as the identifiers of the source. It is partly because I know Goldacre makes regular criticism of the lack of referencing in media reports of science that I was disappointed on a couple of occasions to turn eagerly to the back and not find a citation. These tended to be times when a broad statement had been made – for example on page 75 “A huge amount of research...” does not provide any corroborating references and on page 144 “there is a lot of history here… the field of essential fatty acid research has seen research fraud, secrecy, court cases, negative findings that have been hushed up, media misreporting on a massive scale…[the list continues]” but no notes are offered. If a new edition is produced, please can these be added.

Summary
As I have already said, these are minor (some would say picky) criticisms of an otherwise extremely valuable book. Overall, I believe Ben Goldacre has provided all of us with a toolbox for evaluating sciencey-sounding stories in the media and alerted future scientists to some of the pitfalls they should avoid in the design and reporting of their work. Bad Science would make an excellent resource for post-16 education and I hope to see it adopted as a course text on A level and undergraduate programmes.

Bad Science has a cover price of £8.99 At the time of writing it is available from Amazon for £3.60.

Amongst the major science research journals, Science magazine has consistently been the most prominent in flying the flag for science education. I was very interested, therefore, in an Editorial by Carl Wieman in the September 4th 2009 issue of the magazine. In his piece Galvanising Science Departments, Wieman describes some fairly radical innovations in Science Education currently underway at the University of Colorado and the University of Bristish Columbia. The aim is to adopt evidence-based teaching methodologies with emphasis on the development of scientific thinking and problem-solving skills rather than fact regurgitation.

I have no direct experience of teaching in the USA, either as provider or recipient. I know, for example, that much greater emphasis is placed on the recommended course text in the USA than in the UK, but beyond that I cannot speak with any authority. It does sound like some of the reported innovations are things that have taken place here for some while, such as the addition of specific (skill-centred) learning goals to modules. A cornerstone of the strategy has been appointment of science education specialists, individuals who not only have expertise in their subject discipline, but are also au fait with educational and cognitive psychology studies, a variety of effective teaching strategies and – I note with some mirth – possess diplomatic skills!  The programme is ongoing, the University of Colorado is in the 4th year of an initial six year project and so the full impact of the developments will not be known for some while.

What really struck me, however, was the extent of the commitment at an institutional level, including provision of serious money to fund these changes. Far too many pro-active educators, motivated by genuine desire to improve the learning experience for their students actual receive flak not gratitude. In many cases this is, I believe, because their approach to pedagogy is different to the cultural norm and, as such, moved students (and staff) outside their comfort zone.

So much of education, even at University level, is about recall of a prescribed body of information. This is relatively easy – for both the teacher and the student; it is the mindset that underlies that chirping questions “will this be in the test?” Development of thinking skills demands more from everyone. If innovations that require higher skills are associated with just one module, or even just one academic, then that individual may unfairly receive criticism in feedback from the class.

I believe this is why Wieman is absolutely right when he says “an entire department must be the unit of change”. Depending upon institutional structure, it may even require a larger body – School, Faculty, College – to move together with shared commitment to the new goals. So far at Colorado, 60% of academic staff in three Science departments have embraced the new teaching approaches, impacting 80% of their students’ credit hours. Faculty are reported to enthusiastically discussing teaching as a scholarly activity – that’s surely got to be a good thing. But – it needs time and it needs money.

Wieman C (2009) Galvanising Science Departments Science 325:1181

Professor Melanie Cooper from Clemson University, South Carolina came to Leicester’s Learning and Teaching in the Sciences conference as part of a UK tour sponsored by the Physical Sciences Centre of the Higher Education Academy.  In her talk, Using technology to investigate and improve student problem-solving strategies, Prof Cooper began by drawing an important distinction between problems and exercises.  Often when people set ‘problems’ what they are in fact asking students to do are ‘exercises’, activities designed to train the participants to be able to tackle similar future tasks in a formulaic way.  Problem-solving is about developing a range of skills that will equip students to “address novel situations and arrive at a suitable course of action” (Dudley Herron).  It is not, therefore, about knowing how to crank an equation to get the right answer.

In understanding how students approach problem-solving, there would clearly be huge value in directly observing them throughout the duration of a task.  Such ethnographic research methods, however, have a number of difficulties.  Firstly, the time required for the observations themselves, and all the moreso the subsequent evaluation, is a vast commitment.  Secondly, observations tend to be based, for reasons of practicality, on relatively small numbers of individuals. 

In her education research, Prof Cooper has been able to access a very much larger cohort (several thousand students at Clemson take general chemistry each year) and has got around the need for direct observation of the students at work by exploiting the IMMEX software, developed principally by Ron Stevens at UCLA.  Not to be confused with any similar-sounding floor-to-ceiling cinematic experiences, IMMEX stands for Interactive Multi-Media EXercises. An example of IMMEX use (in the context of genetics education) can be seen in the open access journal Cell Biology Education (see Stevens, Johnson and Soller, 2005).

IMMEX seems to involve some pretty fearsome computing, but I hope the following catches the essence of it.  Students carry out an on-line activity working from a single start-point towards a specific correct answer.  Along the way they can select from a number of briefing sheets, experimental results and other lab data relating to the problem in order to help them to the solution.  Not all of the available information is equally valuable or necessary to complete the task.  The software records the route taken by each student from start to finish (a so called ’search path map’), and uses artifical neural network (ANN) clustering to identify and categorise common strategies.  The technology has to be ‘trained’ by exposure to a large number of examples, and then generates a ‘topological map’, for example a 6×6 grid of ‘nodes’ where similar approaches are clustered together.  Rather than contemplating 36 different approaches, these nodes can then be rationalised into a smaller set of ’states’ representing similar models, in terms of strategies used and/or outcomes achieved.  So, for example, a ‘novice’ strategy might be ineffective (i.e. the student is unable to solve the problem) and/or inefficient (i.e. they visit most or all of the pages before completing the task), whilst an ‘expert’ would take a more efficient and effective route encompassing only the necessary information sources.

The use of ANN allows for helpful categorisation of students’ performance in a particular task.  This can be used to provide them with formative advice on how they might improve their approach.  At this stage a second approach is used to predict and to evaluate the changes in strategies that students make when offered the opportunity to undertake one or more similar tasks.  With sufficient data Hidden Markov Modelling (HMM) can make statistical predictions about the likelihood that students using strategy X will use the same approach again the next time, or whether they will swap to a different tack, and if so whether it will be strategy Y or strategy Z.  The challenge then is whether interventions that we make can move the students on towards a better strategy.

Work by Prof Cooper and others has shown that individual students, be they ‘novice’, ‘competent’ or ‘expert’ at the outset, can improve their competence by repeating activities – but only up to a point.  After five performances, or fewer, none of the participants working on their own exhibited any further improvement in either their ability or strategies employed.  How, therefore, can educators help students to make further refinements in their problem-solving abilities?

Melanie’s evidence shows that an answer lies in group work.  Working with others, particularly those of with different approaches (see below) involves metacognition, i.e. it forces the students into explicit reflection about what they are doing. Tackling a problem collaboratively exposes students to new ways of thinking and/or offers them clarity about why certain approaches are less useful.  What’s more, there is evidence that the improvements made by involvement in group work are retained if the students are subsequently required to work on their own again.

What group arrangements work best? Groupings should be organised by the tutor, not left to the students to choose.  The  maximum group size should be four, and the majority of benefit can be achieved by students working in pairs.  At Clemson, they use the GALT (Group Assessment of Logical Thinking) test as an initial means to identify the type of thinking employed by students – concrete (C), transitional (T) or formal (F), according to the Piagetian model.  Following the GALT assessment, students in Prof Cooper’s research were assigned to pairs according to all possible combinations; FF, FT, FC, TT, TC and CC.  It was clear from the research that there were distinct combinations that afforded greater improvement (to at least one of the pair).  For example, ‘transitional’ students paired with ‘concrete’ improved the most.  Overall female students improve more than males via experience of groupwork, but there was no significant difference based on whether pairings were single sex or mixed gender. Male students, incidentally, improved more as a result of using concept maps than as a result of participation in groups – but that’s probably a story for a different report.

Other talks at the Learning and Teaching in the Sciences conference, by Norman Reid and by Alan Cann,  are discussed elsewhere on this site.