Adjusting “exams” as they move online

Universities across the world are having to adjust to the fact that rooms full of students sitting exams is not an appropriate assessment format for May and June this year. As a consequence, teaching teams are needing to think laterally about how to interrogate students about the learning they have gained from their modules.

celebrity squares

Online meetings have started to look like episodes of the old “Celebrity Squares” gameshow

I am sure many places are way ahead of us on this one, but a few reflections on a recent teaching and learning committee meeting (held via zoom) may be of benefit to those who are just getting going in their thinking about this.

 

  1. Duration of tests and the length of time they are accessible. The amount of time that a test is “live” and the time for which an individual student can respond are not necessarily the same thing. A defined two hour period is not applicable for a number of reasons – including potential timezone differences and connectivity problems. My institution has mandated a “24 hour window” for assessments and it is our understanding at the chalkface (as it was) that this means that they are live throughout that time.
  2. Factual recall questions aren’t going to work. There has, of course, been a long-standing debate about the educational merits of an over-reliance of questions that reward regurgitation of factoids rather than probing higher learning skills. However, a move to remote (ie unsupervised) assessment of students who have ready access to Google* makes this format of question entirely redundant (*other browsers are available).
  3. Students need examples of any new style of questions. A decision that MCQs are not going to be appropriate is only half of the story. Introduction of radically different types of questions is going to require not only production of the actual paper but also additional specimen questions for students who will not be able to draw on past papers for guidance.
  4. Clear and timely instruction. Students are going to need clear guidance before the day of an exam, reiterated in the “instructions” section of the assessment itself. There will be all sorts of practicalities about submission as well as the questions themselves about which students will need advice.
  5. Essay questions will need word limits. For all manner of reasons the standard “three essays in two hours” format is not going to work. More time is inevitably going to mean more words. We all hope that essays represent carefully constructed and reasoned arguments in response to a specific question. Sadly the reality can sometimes be “brain dumps”, in which any material matching identifiable keywords in the title is served up for academics to sift through. A longer time will just allow for more of this unstructured stream of consciousness.
    Even taking a less jaundiced view, a good student is going to be tempted to offer far more material in support of their answer than they would realistically have managed in the typical exam scenario. If we cannot restrict the available time, then another option is to impose a word limit. Having looked at past answers, a suggestion of 1200 words for a 40 minute question (i.e. 30 words per minute) has been floated. The emphasis on “quality over quantity” needs to be emphasised – more is not necessarily better. Of course there may be a minority of student who would have written a longer essay that this, but even they will benefit from tailoring their material as a response to the specifics of the question.
  6. Plagiarism detection, and other “course essay” regulations, are back in play. The kind of measures being considered as “reasonable adjustments” in this unprecedented scenario are much more akin to coursework essays. We aspire to have novel synthesis presented in exam essays, but in the past we would not have penalised faithful regurgitation of material from lectures and other sources. Now, however, there is the very real danger of copy and paste plagiarism from lecture notes, from books and articles, or indeed of collusion between students. The requirement to use plagiarism detection tools is therefore going to be essential. Similarly, students will be able to drop in images taken from sources. Whereas in a constrained exam format we might not have worried about their origins, appropriate citation will need to be factored into marking criteria.
  7. Practicalities about format of paper set and submission requirements also need to be clear. It is not just the content of the questions that need addressing, but also aspects of the delivery of the paper and the safeguards students need to put in place regarding submission. For example it is likely that a paper will be distributed as a Word document, which is actually more accessible than many other potential formats. We know, however, that some elements of layout can be altered during the submission of word documents (eg positioning of images) and so we would probably recommend saving as a PDF before submission (much as we would usually for coursework).

This is not an exhaustive list, and you may instantly spot the flaws in the observations made – if so then do please let me know. I am very conscious that this is prepared in the context of a science program and that other disciplines may see things differently. But I hope these notes will be helpful for at least one or two of you.

 

 

What design tells you about priorities?

I’m intrigued by an apparently accidental case study in design priorities arising from two different posters produced to advertise the same forthcoming lecture. Despite initial appearances, the two posters in the photo both promote a keynote speech by Professor Mary Dixon-Woods on Evidence for Improving Quality and Safety in Healthcare.

IMG_1862

To my mind, the design on the right clearly conveys the content of the lecture (the interesting bit). This crucial information is almost entirely lost in the left-hand version, which massively over-emphasise the series of talks in which the lecture is to be given. It is entirely speculation, but I suspect this emphasis reflects the priorities given in the design brief.

Anyhow, I am looking forward to an engaging presentation from a former colleague.

 

Getting referencing right: applying the 4 Cs

For a variety of reasons, I have been reflecting on principles that undergird good citation practice. So far I’ve come up with “the 4 Cs guide” (I say I’ve come up with them, as I’ve not done any reading on this, but I wouldn’t be surprised if this or something similar has co-evolved elsewhere).

When advising students or colleagues on appropriate organisation of their references at the end of a document, I encourage them to check that they’ve followed the 4 Cs guide:

Correct: Have they cited the correct sources? This might mean drilling back to the first occurrence of an observation (e.g. in the primary literature) rather than a review article. Clearly we don’t want to be encouraging people to cite the original paper if they’ve not read it, but you sometime see statements such as “Smith and Bloggs have shown…” when actually Smith and Bloggs wrote the review in which they discussed the experimental work of Ramone and Farnes-Barnes who made the observation described. Continue reading

  • Awards

  • July 2020
    M T W T F S S
     12345
    6789101112
    13141516171819
    20212223242526
    2728293031