The Higher Education Academy Centre for Bioscience Pedagogic Research in the Biosciences day conference brought together about thirty academics, for the most part Bioscience specialists, who have been involved to educational research. The day turned out to be highly informative and thought provoking. Some on the hoof reflections were collated via Twitter – click this link.

The beginning of July 2008 found me in Haarlem, Holland for the DIVERSE2008 conference. Having been to DIVERSE2006 (at Glasgow Caledonian), I knew the title was an acronym, but whenever I’d been asked what it stood for I replied “I’m not sure, but I do know that the V is for Video and one of the Es is for Education”. It turns out that although I had the gist of it, even this info was inaccurate – DIVERSE stands for Developing Innovative Visual Educational Resources for Students Everywhere, another entry in the pantheon of contrived and convulated names!

I do like the DIVERSE crowd – their backgrounds are as varied as the acronym implies (although Dutch and English are the predominant nationalities). What the delegates have in common is an interest in the use of video technologies in education. Applications can be quite varied; there’s a strong video-conferencing strand (which isn’t really my thing) but I picked up sessions in all four of the parallel themes: pedagogy and assessment; tools and content orientated applications; projects and cases – implementation and sustainability; and people and technology – societal aspects. This year’s event included a demonstration of a live cello masterclass with the instructor in Holland teaching a pupil in America via Internet2, a high-quality connection.

 

Live cello masterclass conducted over Internet2

As befits a video-centred conference, all of the presentations from the event are now available online. I have also added a draft of Sharing the vision: exploiting Web 2.0 technologies in promoting the use of multimedia in bioethics education, the paper I submitted for the Proceedings booklet.

Although the conference was hosted at InHolland University of Applied Sciences in Haarlem, the accommodation was a short train-ride away in the seaside town of Zandoort. If you are thinking Rotterdam, think again; Zandoort has a great beach (no, really) and we were blessed with fantastic weather for the conference dinner at Skyline 13, the Australian Beachbar.

Conference dinner at the beach

 

Not sure Aberyswyth will look like this (mind you, it's hard to believe that's Holland!)

On 8th April 2008, the University of Leicester played host to conference organised by the Centre for Bioscience of the Higher Education Academy (Editorial note: apologies it took so long to get this post up – it was an excellent day conference so I hope you’ll find the material still relevant. More notes can be seen at the official Centre for Bioscience summary of the event).

Cooking the books?
First up was Fiona Duggan from the JISC Academic Integrity Service. Fiona started by highlighting recent discussion in the media about Delia Smith’s book How to cheat at cooking – is it really “cooking” to use frozen mash? Computer games have built in capacity to “cheat”. Are these symptomatic of a change in the acceptability of cheating in society?

Fiona then posed the question “How are students to know what is acceptable practice?”. She cited a post to Deliaonline where a beginner was brave enough to say they didn’t understand what “browning” mince meant. Similarly, students may not know all the ‘obvious’ things we assume that they do. They may have a different take on the ‘rules’; for example, some students argue that referring to online essay banks was simply a way to see model answers since staff had failed to provide examplar material outlining what was expected of them. This may involve some fairly remedial work (from a lecturer’s perspective) and several academic colleagues may feel under-equipped to tackle this sort of thing. Involving librarians and/or information technologists may be useful.

What do instructors and institutions need to do to tackle plagiarism? Several universities have adopted an Enforcement model, punishing wrong-doing, but this does not work if students don’t know what it is that they’ve done wrong. There are also worries about inconsistent practice by institutions across the HE sector. This is only treating the symptoms not the cause.

A second strategy builds plagiarism around an Ethics model – based particularly on the honour-code model in the USA. This is a long-term project, it’s about changing the culture through the institutions – note by comparison how many years it took to change the drink-drive culture in the UK. No empirical evidence on honour codes is available yet from the UK, although work is under way on this approach, particularly at Leicester and Liverpool.

A third strategy could be described as Engineering, that is re-designing our assessments to make them harder to complete by cheating. Don McCabe of Rutgers University has been particularly active in this approach.

Fiona went on to discuss the phenomenon of ‘contract cheaters’, i.e. people who pay for their assignments to be completed by someone else. She highlighted a study on computing students who post their assignments online, with other students bidding to do the work. The most worrying thing was that nearly 50% of students who were doing this did so on more than one occasion. Jude Carroll chipped in at this point to say that it wasn’t simply a last-minute panic – students were posting assignment requests on sites such as RentaCoder shortly after they were set.

We have to look at the way we design our assessments to reduce the possibility. This frequently prompts howls of disapproval from colleagues “It’s such a lot of work” – yes, it will involve more work, but their are lots of ideas out there that you can pinch! People are developing all sorts of engaging and interesting activities, such as games to teach appropriate use of sources and citation practice.

Conclusions from Fiona’s session: students need to be discouraged from using “frozenmash and tinned mince” approaches to their studies. At the same time, lecturers need to be discouraged from using a Blue Peter “here’s one I used earlier” approach to setting assessments.  We need to emphasise that we are involved in the process more than the product. We should aim to help students move from being ‘don’t know how’s to being confident users of information. She finished with a quote from the Times Higher Education Supplement (as it then was) from March 2007: “learning practical chemistry shouldn’t simply be about ‘taxing’ the student to follow a long and complex recipe. It should be about making the student think about the techniques they are using and why their chosen technique is superior for that compound”.

Fiona’s slides can be seen here.

Electronic detection of plagiarism
Following on from Fiona, Jo Badge, web resources development officer at the University of Leicester, talked about use of the TurnitinUK software and Safeassign, a second package designed for integration with the Blackboard VLE system. Turnitin was originally introduced into the UK in 2002, having been developed previously in the USA. The system operates by looking for matching text, in a manner analagous to genome matching software. String-searches, of course, do not in themselves prove or disprove plagiarism. Submitted work is held on a national database, in perpetuity. Turnitin can be integrated with Blackboard and responsibility for submission can be devolved back to the student. Turnitin has been in use at the University of Leicester since October 2004. Within biosciences it is routinely used for screening of essays in Years 2 and 3, and laterly for some Year 1 assignments. For final year projects, students are asked to remove their references and images prior to electronic submission; the former to reduce cross-match, the latter to reduce file size. Students can use the quickview feature to check they’ve submitted the right file, and they receive an e-mail to say that submission has been successful. This acts as a receipt and offers reassurance to students concerned that their precious work may have disappeared into the ether never to be seen again.

Safeassign is a buyout of an earlier product Mydropbox. It comes “free” as part of a very expensive Blackboard service, and is likely to become more popular with the roll-out of Blackboard version 8. There are a number of unsatisfactory features about the package at the moment; e.g. students don’t get a confirmation e-mail, and it uses Microsoft windows live search, which is not a brilliant tool. In addition it ppdates searches over time, incorporating materials added more recently in the comparison. Turnitin reports, in contrast, are fixed – if you determine (via the software) that the submitted work is a 20% match to another essay, let’s say, that number won’t have changed next time you come back to Turnitin, but it might if you are using Safeassign and this may be confusing. Side-by-side view, one of the attractive features of Turnitin, is limited in Safeassign and there is no option to remove quotes/references. The description of the comparison as “% probability” is rather confusing.

Jo went on to elaborate about the context at Leicester. The School has approximately 100 taught postgraduates per year and 600 undergraduates. In November 2004 a retrospective pilot study was carried out using samples of work which has coincidentally been submitted electronically in the previous year. This was followed by an initial trial involving 14 modules. These initial studies threw up examples of cut and paste copying, patchwork writing (i.e. sewing together of multiple sources, largely unaltered), close paraphrasing and some collusion. There didn’t seem to be much inter-year copying, although interestingly she has now seen some examples of students reusing their own work in different contexts in different years.  Jo reported the following number of cases of plagiariam detected against the number of items test in that year:  2003-04, 11 out of 97; 2004-05 34 of 513; 2005-06, 21 of 1430; 2006-07 63 of (unknown). Scanning of first year work has generated larger numbers, but it is important that the issue arises at that stage since Year 1 does not ‘count’ towards the degree and it allows students to learn about acceptable and unacceptable practice before it has a detrimental effect on their degree performance. A sliding scale of penalties has been introduced based on the severity of the case (how much has been copied) and whether it is a first offence or not.

Jo finished with a number of ‘live’ issues concerning the use of plagiarism detection software – should you allow students to self-check their own work with the software prior to formal submission? Leicester currently does not allow this, other places do. A low percentage match score does not guarantee an essay is original – copying may have come from a password-protected source not available to the comparison software, or it mat have been commissioned by the student. There also needs to be a certain flexibility about what constitutes plagiarism – you cannot apply a strict “above X% is plagiarism, but less is ok” rule, since some percentages, e.g. 10% may be from a single paragraph lifted verbatim or lots of short phrases of technical language which it would be hard to rephrase. The key thing is that a consistent approach is taken.

 Jo’s slides can be seen here.

Teaching students what plagiarism is to prevent it
Next up, Maureen Dawson and Joyce Overfield (Manchester Metropolitain University) talked about a project started about five years ago to investigate students and staff understood about plagiarism. This, in turn had led to the production of guidelines for their Department. I won’t say too much more about it here as this project has been written up in the Centre for Bioscience journal Bioscience Education. In essence, they presented students with MCQs and case scenarios, developed from genuine examples, and asked them which examples they considered to be guilty of plagiarism. They stressed the need to regularly reinforce the rules via an online version of the activity (password protected).

Maureen and Joyce’s slides are available here.

Improving scientific literacy to prevent plagiarism
Dorothy Aidulis began by setting the scene for work she has been involved in at the University of Glasgow.  She believe that we not only need to clarify what plagiarism is, but also to help students with their understanding of how science works. She has therefore developed scientific writing workshops as a mean to improve scientific literacy.

Dorothy showed a typical definition of science, drawing our attention to the active processes: “understanding… making observations… collecting data… explaining them…” That being the case, why does our teaching of science so often overemphasise regurgitation of a “body of knowledge” at the expense of elaborating the ”process of study”?

In Dorothy’s workshop, she starts by issuing students with post-it notes and gives them one minute to write their own definition of plagiarism. After a talk about University regulations and the reasons for those regulaitons, they move onto a set of activities. In the first students are given example texts and highlighter pens and essentially act as “manual turnitin”, picking out sections that they believe are guilty of plagiarism.  They then move on to another activity, on summarising texts in their own words, before thinking about appropriate referencing practice. Afterwards, she collects feedback using a “what was something you did not know – something you knew already – something you’ve changed your mind about.” model.  

Dorothy’s slides are available here.

Using course and task design to deter students from plagiarism
Jude Carroll, Deputy Director of the Assessment Standards Knowledge Exchange (ASke) CETL at Oxford Brookes University led the day’s interactive session. In her workshop, Jude was keen to get us thinking how we can reduce the amount of plagiarism that occurs by altering the format of assignments we set. She emphasised that it really needs a set of fresh eyes, not the module convenor, to make a course plagiarism-proof. We need to move from “find it” tasks to more “make it” assignments – Google exists, it’s not going away, so setting relatively trivial knowledge-unearthing activities is an invitation for students to copy sources. One example she gave, replace “write an essay on smoking and public health” with “Find three ’stop smoking’ websites. Create criteria to judge which will best improve public health, rank them and justify the ranking”.

Jude asked us to consider why students commit plagiarism. It is not just wilful desire to cheat, some (possibly most) is inadvertent. Misunderstanding (not knowing the rules), misuse (knowing the rules but not how to fulfil them) and misconduct (knowing the rules and knowing how to fulfil them, but not bothering) can each be the underlying reason for plagiarism. Jude encouraged us to think about our role as educators in “academic apprenticeship” and how we can both ‘design out’ bad practice and ‘design in’ appropriate skills. This includes building opportunities for early diagnosis and feedback on following/failing to follow academic rules and conventions. She stressed the need for excellent written support and guidelines which students can refer back to after a session (and not just online). If we want students to both to read our support materials we need to make sure they look good – this conveys that we are serious about the issues, and encourages them to be invest similar attention. This led into discussion, more generally, of the importance of modelling by academics; do we, for example, apply the sort of citation standards we expect from students when we are producing our module handouts and our powerpoint slides? (in my notes from the event I have a comment here about ’sheep-dipping’, but have no idea what it means! Perhaps it was to do with getting to the student before they get infected – can anyone who was there help me out?!?)

When it comes to designing out plagiarism, it is important to start early – habits and behaviour rapidly get ingrained and if we feed students a consistent diet of “find and display” tasks then this is what they’ll learn to deliver. If we persist in offering those kind of regurgitation assignments we mustn’t be surprised when we end up with 10% of our classes on the offenders register! Rather than just moaning about the poor planning skills of the modern student, we need to acknowledge the weakness and find ways to tackle it; for example, design in compulsory stages in the task, make it necessary to see a ‘learning log’ regularly and sign it. As mentioned above, review the assessment criteria, emphasise the process not the product.  Reward good referencing practice, don’t just penalise poor examples. Make work public and owned, introduce an element of peer review. Vary the tasks between different academic years, rotate them on a cycle of at least three years duration.

Judes’s slides are available here.

Swapshops
The event finished with a series of shorter presentations by conference delegates. Jon Scott (University of Leicester) kicked off with Policy evolution and the elusive grail of consistency.
Jon noted that there had been changes in code of conduct and regulation over the years.  There were a variety of drivers in this process: differences in student background, advances in technology with students responding faster, leaving institutions as gamekeeper playing catch-up, and pragmatism. It was an important goal to seek consistency in the application of penalties. Given the number of cases or possible cases of plagiarism being unearthed by plagiarism software, it would be impossible to host the traditional panel meeting for every offence. Biological Sciences at Leicester has therefore moved to a fixed penalty ‘on the spot’ fine for first offences, akin to receiving a letter about a speeding offence. Students can appeal, but 99% take it on the chin, although this is not distinguishing those who are tacitly admitting the offence as against those who can’t be bothered to appeal. Both students and staff need further training, the latter to tackle residual subjectivity in the initial reporting of offences.  Jon’s slides can be seen here.

The Second swapshop presentation was by Viv Rolfe from De Montfort University, who shared some of her work Using Turnitin for essay drafts and final submissions. - mandatory for year 1 from Sept 08, short notice! Draft and final. Fun sessions, 3 hours – a piece of deliberate work, missed items? view a draft once – 93 students: 31 plagiairism, 1 collusion. No attempt to correct, blatant – those who did do change, breaking strings, word-swaps, no genuine effort to synthesise. Not improving academic practice! Viv’s slides can be seen here.

Finally, Stuart Johnson demonstrated Don’t Cheat Yourself, his series of Subject-specific online tutorials to help students understand what plagiarism is and how to avoid it. The first tutorial in the series was an adaptation of the paper-based exercise we had developed and published in Journal of Biological Education a few years back. Since then the core activity has been tailored to suit a range of different disciplines by the inclusion of subject-specific examples. Stu’s online tutorials are available here.

The Higher Education Academy Centre for Biosciences event report is available here.

“Marking, remarking and meaningful learning: an assessment and feedback seminar” was held at the University of Leicester on April 4th 2008.  The event was organised by the Assessment and Feedback Working party of the University’s Student Experience Enhancement Committee and was attended by about 60 members of the academic community. The following are personal reflections and things that I took from the day.

Smartie-pants
The first presentation was given by Jon Scott, Director of Studies in Biological Sciences at the University. Jon’s cryptic title “How the baby got the Smartie” actually drew analogies between his research work on development of motor coordination skills and effective use of feedback. The ability of a baby to pick up a smartie from a flat surface is apparently a developmental landmark (presumably there are healthy options now available for choco-phobic parents). Research on brain activity whilst learning this task has shown that neurons are fired by failure to achieve the task, i.e. whilst the infant is self-feedbacking (is that a word?) . It knows what it is expecting (bright, interesting-looking object in mouth) and feedback modifies performance until it gets it. Once the task has been mastered, apparently, the relevant neurons go silent.

 Jon then applied this observation to the way students use (or ought to use) assessment and feedback. He stressed that feedback is an active process for assessor and learner and that feedback can show both what is ok, and what is needed for next time – hence we need to encourage reflective behaviour in students – they need to review their work in the light of feedback received to work out how to do better next time.

Why did I get 37%?
Next up was Brenda Smith from the Higher Education Academy. Brenda’s theme was Assessment for Learning: why did I get 37%? AfL is a topic we’ve considered previously at the Journal of the Left-handed Biochemist. The title of Brenda’s session reflects a comment made by a student in a research project; their tutor was reported to have refused point-blank to enter into discussion on the matter.

Brenda began by reflecting on the growing evidence from QAA subject reviews, the National Student Survey, etc that all is not well in the world of Assessment and Feedback. The problems noted are familiar to anyone who has taken an active interest in this process: too much summative feedback, not enough formative; feedback coming too late to be useful; inconsistency in assessment practice both within and between institutions. Referring to data from the 2006 and 2007 National Student Surveys, Brenda highlighted that the lowest scores (51% for both questions, in both years) were in response to the statements “feedback on my work has been prompt” and “feedback on my work has helped me clarify things I did not understand”.  These are serious issues if feedback is to give relevant guidance for improvement in a timely fashion.

Tackling a number of ‘myths’ about assessment and feedback, Brenda questioned whether high failure rates on some courses could be defended as “maintaining high standards” as some institutions might try to do so. The thorny issue of feedback on exam performance also came up – the more students pay for their courses, the more they feel they are owed it. I certainly have some sympathy with the need to offer students training and advice on taking exams, including the opportunity to receive both peer and tutor feedback – indeed we have run an exercise with precisely this aim over a number of years (see ‘You have 45 minutes, starting from now’: Helping Students Develop their Exam Essay Skills). I do, however, worry that this genuine need, and entitlement, would descend into the type of routine calls for remarking that are now rife with A level courses.

Turning to the need for consistency (but stressing that consistency was not the same as conformity), Brenda then showed some discrepancies between the different balance of coursework to examination ratios in a number of departments at a University “not dissimilar to Leicester”. One Department highlighted seemed to assess students in their second year entirely on the basis of exams. Given the lack of feedback students in general receive about exam performance, do these students therefore receive any feedback at all for the whole of that academic year? We were challenged both to know more accurately about assessment practice in other Schools and Depts within the institution, but also to actively seek out examples of best practice elsewhere (e.g. via joint awaydays).

Why do we assess? Brenda gave four reasons: certification (i.e. to show someone was fit to practice), quality assurance, learning and sustainability. She argued that we have perhaps overfocussed on the first two and not enough on the others. Regarding sustainability, for example, do we do enough to equip our students to be sustainable learners themselves, and also to be future givers of valuable feedback to others. Do we encourage students to read each other’s material and offer one positive and one negative comment; to develop a habit of critical assessment?

To make the best use of this opportunity, students will need some training to know what they are looking for. Begin simply, e.g. give them three pieces of work – one good, one average and one weak – can they tell which is which, and why? What feedback would they offer the authors of each piece of work? Moving on from that, what would they feel like if they received the feedback they had just written?

Finally, Brenda pointed us to the Student Enhanced Learning through Effective Feedback (SENLEF) pages on the Higher Education Academy website. She drew our attention particularly to the card-sorting activities and to the seven principles of good feedback practice.

How was it for you?
Next up was Aaron Porter, currently a Sabbatical Officer in the Students’ Union at Leicester, and recently elected  to the national leadership of the National Union of Students. Aaron began by showing a series of voxpop interviews featuring about ten students answering a set of questions: What is feedback? What is good feedback? What do you do with the feedback given to you? Does your personal tutor help you with feedback? How has feedback you received helped you to learn? What would be useful feedback to you? The views expressed were interesting, but without offering and particular fresh insights. I was starting to get really cross when all of the students talked about how carefully they re-visit the work after they’ve received feedback and was much relieved when Aaron also expressed his skepticism about how representative this was.

In fact I was generally impressed by Aaron’s presentation and he raised a number of thought-provoking issues. He considered the timing of assessments, the balance of different types of assessment, the usefulness of feedback and the use of appropriate technologies for the 21st Century. His thoughts on timing were probably the most interesting. Communicate with colleagues more effectively to spread out the load – this allows students to actually spend more time on each piece of work, spreads the marking burden for STAFF and has relevance for the mental health of students. The first of these wasn’t new, the second I actually disagree with (I LIKE marking coming in at certain times as it leaves other time periods free of marking) but it was the third point that was the fresh insight. Of course, once aired it makes a lot of sense; multiple deadlines crammed into a short period of time, coupled with financial worries, the burden of parental expectation (especially if they are paying) can be highly detrimental to a student’s mental well-being. He also put in a general plea for earlier notification of exam timetables to facilitate better planning, which is something staff would wholeheartedly echo.

Commenting on the National Student Survey, and the apparently poor satisfaction scores, Aaron made the point that if there is an approx even split between coursework and exams then the fact that most students prefer one or the other means that no-one is ever going to be offering top marks for assessment satisfaction. Good point. He also stressed the need to make students aware that they are receiving feedback in a variety of formats, it is not only written comment and advice from their personal tutors.

Whilst recognising that there are logistical problems associated with feedback on exam scripts, he added his voice to concerns that the status quo is unsatisfactory from an educational perspective. Having said that, Aaron did also query the appropriateness of exam essays as an assessment format at all. Students, he suggested, are increasingly calling for ‘real world’ relevance of the tasks that are set for the. When will someone in employment ever be asked to write on a topic (as opposed to word process), in a linear start-to-finish manner, for a period of an hour, without reference to source materials or the internet? Is it time, he wondered, for students to be allowed to use computers in exams?

Personally, I’m not sure this is a runner – unless there was some kind of auto-archiving as they went along, I can just see lots of people crying into their keyboards after two and a half hours complaining that they’ve accidentally pressed delete and lost all of their work. I’m also not taken with Aaron’s final suggestion for online tracking of coursework so that students can know where in the marking/second-marking process their work has got to, in the same way that you might track the progress of an online delivery. It strikes me this will involve a huge administrative burden for very little net gain. It was funny to hear it so soon after Alan Sugar had mocked the 24-hour hotline for laundry monitoring on this week’s The Apprentice –  they seem like two peas in a pod.

More for less?
The last presentation was by Phil Race. This is the second time I’ve been to a session led by Phil and hearing him again reminded me how many practical tips and tricks I’d taken and applied in my teaching after our previous encounter. He is also very generous with his resources via his website, www.phil-race.com. This time around his topic was How can we get better feedback to more students in less time?

One of the points that came across most strongly from Phil was the need to separate feedback from the return of marks; in the table discussion before the talk he went as far as to say that “it’s unethical to give students a mark at the same time as feedback”. Feedback is most useful when it is given within 24 hours of a submission deadline, since the material will be fresh in the minds of all the students. Clearly we can’t be expected to have marked or even read all of the work in that timescale, but there is plenty of generic feedback we can give – the sorts of things that we end up writing time and again on student scripts (which allows the individual feedback we subsequently give to be all the more targetted).

The predictable problem with this approach was raised – how do you cope with late submissions? Phil was clear on this issue too – why should the 97% of student who submitted on time miss out on prompt and timely feedback just because of the 3% who haven’t made the deadline? The problem is sidestepped if an alternative but equivalent Assessment B (albeit deliberately on a slightly less appealing topic) is set at the same time of the original Assessment A. If a student misses the deadline for Assignment A then they do B; that way they get fair treatment if the deadline was missed for a genuine reason.

If you give feedback before you give the mark it also allows for an additional carrot to ensure students engage fully with the comments and advice that we’ve offered – if, on the basis of the feedback given, they come up with a mark within 5% of the mark you awarded they can have whichever is the higher mark. After reading your comments, more than 90% of students will be within that range of your score – you can then target discussions with those who have significantly mis-read the merit of their work and examine where the false perception has arisen.

The other main point I took away from Phil’s contribution – and it was from the round-table discussion rather than the talk – was the merit of offering students a proforma for recording the feedback they have received and the use to which they have put it. The form can be stored in their portfolio of PDP evidence, it can be a tool for aiding genuine interaction with their feedback to make it feedforward, and it may allow them to identify repeat themes coming up from several different markers. Of course some may not choose to use the proforma, and others may simply ‘play the game’ without really entering into the spirit of the exercise, but for those who do make the most of the opportunity it may be a valuable addition to their learning experience.

To summarise:
Overall, this was a useful day – nothing ground-breaking, but some helpful reminders and some new food for thought. As is so often the case, the big frustration is one of ‘preaching to the converted’ – I suspect that the assembled staff are probably amongst those who are already the most conscientious about the feedback they give. It was also interesting to participate in Alan Cann’s experiment in liveblogging using Twitter (with hashtagging) - a valuable way to share and capture insights in 140 character chunks.

The fact that you are reading this blog entry at all means that you are already engaging with Web 2.0, which has been defined on Wikipedia as “a perceived second-generation of Web-based services such as social networking sites, wikis, communication tools, and folksonomies that emphasize online collaboration and sharing among users”.  In the third talk at the University of Leicester Learning and Teaching in the Sciences conference on 23rd May 2007, Alan Cann raised the potential impact of Web 2.0 technologies in science teaching.

Dr Cann began with the definition of Web 2.0 given in the previous paragraph, and illustrated how broadly we have come to accept interactive aspects of the web with reference to Amazon.  Ostensibly an online shop, Amazon offers us the opportunity to review the goods on sale, even allowing us to give critical reports.  Similarly, we are invited to rate the performance of sellers for whom Amazon has acted as middleman. 

Alan highlighted the fact that, when asked to write an essay, the default strategy of today’s student is to turn to Google and Wikipedia.  We may not like it, but this does not change the reality and, setting the pattern that was to run throughout his presentation, Dr Cann challenged us to think about ways that we can work with and develop the students’ study habits rather than fighting against them.  So, for example, we should teach students how to use Google more effectively to obtain the best quality information, rather than simply chastising them for using such a shoddy tool and brow-beating them into using the ‘proper’ searches.  This does not mean that we abandon training sessions on PubMed, Web of Science and the like, far from it.  We start with Google and move on to the more professional tools as an extension of good practice.

Against that backdrop, what is the place of wikis, blogs, podcasts and the like in the teaching of science? Alan suggested that, used appropriately, these Web 2.0 technologies can be particularly helpful in engaging the ‘long-tail’ of less able and less motivated students who do not respond well to the traditional approaches.  Clearly we need to adapt our writing style to be appropriate to the medium – the academic journal genre is not appropriate for blog entries which must be more bitesized and engaging.

What about podcasts and ‘viral’ video?  Dr Cann shared some insights from his personal experience and research projects conducted over the previous couple of years.  Alan has been developing blogs, podcasts and online video for the public understanding of science (specifically microbiology), for using in teaching statistics to first year undergraduates at the University of Leicester, and to share his virtual frogroom with fellow tropical frog enthusiasts. In doing so he has gathered both statistical data and qualitative comments from users concerning the relative merits of different approaches.  His observations included:
(1) A general dislike for the ‘push’ model of subscription via RSS feed, people prefer to ‘pull’ material to their computer as and when it looks of interest to them.
(2) Students are happy to listen to ‘work’-related podcasts on their computer, but reserve use of their mp3 player for ‘entertainment’.
(3) More students watch online videos via YouTube, and the like, than listen to podcasts.

Dr Cann finished by reiterating the point that this is not a call to ‘dumbing down’ and that the intention was to offer Web 2.0 resources to students in addition to traditional approaches.  The materials produced must remain academically robust, but should be offered in a format that is comfortable and familiar for 21st Century undergraduates.

Professor Melanie Cooper from Clemson University, South Carolina came to Leicester’s Learning and Teaching in the Sciences conference as part of a UK tour sponsored by the Physical Sciences Centre of the Higher Education Academy.  In her talk, Using technology to investigate and improve student problem-solving strategies, Prof Cooper began by drawing an important distinction between problems and exercises.  Often when people set ‘problems’ what they are in fact asking students to do are ‘exercises’, activities designed to train the participants to be able to tackle similar future tasks in a formulaic way.  Problem-solving is about developing a range of skills that will equip students to “address novel situations and arrive at a suitable course of action” (Dudley Herron).  It is not, therefore, about knowing how to crank an equation to get the right answer.

In understanding how students approach problem-solving, there would clearly be huge value in directly observing them throughout the duration of a task.  Such ethnographic research methods, however, have a number of difficulties.  Firstly, the time required for the observations themselves, and all the moreso the subsequent evaluation, is a vast commitment.  Secondly, observations tend to be based, for reasons of practicality, on relatively small numbers of individuals. 

In her education research, Prof Cooper has been able to access a very much larger cohort (several thousand students at Clemson take general chemistry each year) and has got around the need for direct observation of the students at work by exploiting the IMMEX software, developed principally by Ron Stevens at UCLA.  Not to be confused with any similar-sounding floor-to-ceiling cinematic experiences, IMMEX stands for Interactive Multi-Media EXercises. An example of IMMEX use (in the context of genetics education) can be seen in the open access journal Cell Biology Education (see Stevens, Johnson and Soller, 2005).

IMMEX seems to involve some pretty fearsome computing, but I hope the following catches the essence of it.  Students carry out an on-line activity working from a single start-point towards a specific correct answer.  Along the way they can select from a number of briefing sheets, experimental results and other lab data relating to the problem in order to help them to the solution.  Not all of the available information is equally valuable or necessary to complete the task.  The software records the route taken by each student from start to finish (a so called ’search path map’), and uses artifical neural network (ANN) clustering to identify and categorise common strategies.  The technology has to be ‘trained’ by exposure to a large number of examples, and then generates a ‘topological map’, for example a 6×6 grid of ‘nodes’ where similar approaches are clustered together.  Rather than contemplating 36 different approaches, these nodes can then be rationalised into a smaller set of ’states’ representing similar models, in terms of strategies used and/or outcomes achieved.  So, for example, a ‘novice’ strategy might be ineffective (i.e. the student is unable to solve the problem) and/or inefficient (i.e. they visit most or all of the pages before completing the task), whilst an ‘expert’ would take a more efficient and effective route encompassing only the necessary information sources.

The use of ANN allows for helpful categorisation of students’ performance in a particular task.  This can be used to provide them with formative advice on how they might improve their approach.  At this stage a second approach is used to predict and to evaluate the changes in strategies that students make when offered the opportunity to undertake one or more similar tasks.  With sufficient data Hidden Markov Modelling (HMM) can make statistical predictions about the likelihood that students using strategy X will use the same approach again the next time, or whether they will swap to a different tack, and if so whether it will be strategy Y or strategy Z.  The challenge then is whether interventions that we make can move the students on towards a better strategy.

Work by Prof Cooper and others has shown that individual students, be they ‘novice’, ‘competent’ or ‘expert’ at the outset, can improve their competence by repeating activities – but only up to a point.  After five performances, or fewer, none of the participants working on their own exhibited any further improvement in either their ability or strategies employed.  How, therefore, can educators help students to make further refinements in their problem-solving abilities?

Melanie’s evidence shows that an answer lies in group work.  Working with others, particularly those of with different approaches (see below) involves metacognition, i.e. it forces the students into explicit reflection about what they are doing. Tackling a problem collaboratively exposes students to new ways of thinking and/or offers them clarity about why certain approaches are less useful.  What’s more, there is evidence that the improvements made by involvement in group work are retained if the students are subsequently required to work on their own again.

What group arrangements work best? Groupings should be organised by the tutor, not left to the students to choose.  The  maximum group size should be four, and the majority of benefit can be achieved by students working in pairs.  At Clemson, they use the GALT (Group Assessment of Logical Thinking) test as an initial means to identify the type of thinking employed by students – concrete (C), transitional (T) or formal (F), according to the Piagetian model.  Following the GALT assessment, students in Prof Cooper’s research were assigned to pairs according to all possible combinations; FF, FT, FC, TT, TC and CC.  It was clear from the research that there were distinct combinations that afforded greater improvement (to at least one of the pair).  For example, ‘transitional’ students paired with ‘concrete’ improved the most.  Overall female students improve more than males via experience of groupwork, but there was no significant difference based on whether pairings were single sex or mixed gender. Male students, incidentally, improved more as a result of using concept maps than as a result of participation in groups – but that’s probably a story for a different report.

Other talks at the Learning and Teaching in the Sciences conference, by Norman Reid and by Alan Cann,  are discussed elsewhere on this site.

The annual Learning and Teaching in the Sciences event at the University of Leicester was held on May 23^rd 2007.  Three invited speakers brought very different insights into the effective communication of science. This entry focuses specifically on the first of the presentations.  Other talks, by Melanie Cooper (Clemson University, USA) and Alan Cann (University of Leicester) will follow in subsequent posts.

Norman Reid (Professor of Science Education, University of Glasgow) addressed the subject of the ways we can maximise the impact of our teaching by taking into account scientific studies into the factors that influence learning.  I had heard Norman speak previously on the subject of pedagogic research methodology (he has written a very useful booklet on the subject on behalf of the Physical Sciences Centre, Higher Education Academy).  I had high expectations, and I wasn’t disappointed. 

Early on in his talk, Norman emphasised the importance of Working Memory Capacity (WMC), in other words how many ideas are we capable of holding in our short-term memory at any one time.  In an exercise reminiscent of the 1980s gameshow The Krypton Factor, we were asked to convert a date into single digits, and put them in numerical order (without writing them down).  So, for example, 7th April 96 would be 4-6-7-9.  As the number of digits involved increased, the capability to solve the puzzle diminished.  If, therefore, we are presenting students with more distinct pieces of information than they can cope with (in other words, if the information load of our teaching exceeds their working memory capacity, then this is going to have a detrimental impact on their learning. Rather than a linear decline in success as information load increases, there is a sudden collapse in performance.  For most people, the WMC seems to be about 7 items.  This number varies from person to person and, it seems, we can do little to change it. Norman mentioned grouping strategies and pattern recognition as ways in which we can carry more bits of information than our WMC, but this is making the best of what we’ve got, not stretching the underlying capacity.  He didn’t specifically discuss mnemonics, but I guess these are an example of a grouping strategy.

The place of WMC in an information processing model was then fleshed out.  In addition to Working Memory and Long-Term Memory, an important role is also played by a Perception Filter.  I took the latter to be a subconscious self-recognition of the number of bits of information you can cope with.  To draw an analogy (my own, apologies to Prof Reid if I’ve got this wrong!) – if you were the captain of a ship, you would know how much cargo you can carry on board.  You would decline extra items, even if they were on offer.  In similar vein, a perception filter allows you to ‘know your limits’ – there may be extra information on offer, but when you know you are in danger of overload you engage mechanisms that stop taking too much on board, lest the ’ship’ sinks.  I guess, by extension of my image, there is benefit in being able to distinguish valuable cargo from junk, which is probably one reason why our previous experience and our long-term memory influence the effective working of our perception filter.  Norman used the term field dependency for the ability to see what is important, to distinguish the ‘message’ from the ‘noise’.

Pushing my analogy to its conclusion, I suppose our role as educators would equate to the port authorities or harbour master.  We need to be aware of the number of fresh bits of cargo we are offering to our students, and ration their delivery so that we reduce the risk than anyone tries to set sail with too much on board (suspicion I pushed that too far – Ed). 

In the next phase of his talk, Prof Reid moved on to consider the idea of pre-learning. At its most simple, this might be starting a lesson or a lecture with a couple of minutes of reflection (“ok, who can remember what we discussed last time?”).  This is all about making connections between different nuggets of information.  Having a list of review questions up on the screen at the start of the lecture and asking students to work through them in pairs was a recommended model.  This might be extended to a formal short activity or exercise taking place before a major lecture or laboratory practical to draw attention to what are going to be the main points, thus equipping the students more effectively to distinguish message v noise.

Once again, these ideas rang true for me.  I know I’m not alone in seeing that one of the downsides of modularisation has been the compartmentation of knowledge.  Students do not necessarily see the connections between the different teaching within a module and less so between units.  It is one of the roles of the educator to make explicit the links to previous and future teaching, since they (hopefully!) have a better grasp of how the bits fit together.

Prof Reid emphasised that reducing the working memory load was emphatically not a call for ‘dumbing-down’.  The challenge is not to throw out the hard topics, but rather give conscious consideration to the order in which material is covered, to connections between material more overt and to break down complex items into more comprehensible sizes.

As the session moved towards questions, much of the discussion focussed on the research methodologies employed to produce the scientific data undergirding these views.  In particular, delegates and speaker alike expressed a frustration that the demands for ‘fairness’ meant that it was becoming very difficult to conduct proper comparisons between groups experiencing different teaching.  True, crossover studies (where group A is taught using method X and group B is taught using method Y, and then the two groups are swapped over for a second phase of teaching using the other method) can partially fulfil this need, but there are plenty of occasions when this is not truly feasible.  In consequence, many of the most informative studies have been performed outside of the UK.  Food for thought.