The promise of anonymity can undermine the value of education data
From time to time I am asked to comment on other people’s unpublished research. As part of the evidence offered in the manuscript, it is quite common to see analysis based on anonymous questionnaires conducted before and after a pedagogic intervention. In this post I want to raise some concerns about the significant limitations that arise from the unnecessary anonymisation of survey data.
Why offer anonymity?
Firstly, however, it is worth examining the allure of anonymity. From conversations I’ve held with colleagues, the main attraction of anonymisation is the perception that removal of identifiers will free participants to provide full and frank contributions, secure in the knowledge that there can be no personal come-back.
I want to argue here that there are important research benefits from *avoiding* complete anonymity, except in the vanishingly rare occasions where it is vital that contributors cannot be recognised.
1. Keeping identifiers allows for richer analysis. If you can match pre- and post-intervention data it is possible to report on changes relating to individuals which may have been masked by analysis of the cohort as a whole.
2. Keeping identifiers guards against inappropriate comparison of whole cohort data. There is a temptation to take all of the available pre-intervention data and compare it with the complete set of post-intervention data, thereby ensuring that a minimum of data is “wasted”. I believe that this is wrong-headed and to illustrate this point, consider the following scenario in education research. Continue reading
Comparing 'before' and 'after' data needs some identification
When undertaking educational research you often want to know how an intervention has affected a cohort, and ideally to be able to drill down into the data to see the impact on individuals. In order to match pre-and post- activity surveys, some kind of identifier is required. You could ask the students to put their names on the forms, but they may have concerns that this will have ramifications for their coursework. What else you could do?
There are a range of semi-anonymised labels you could use. At various times in my own work I’ve used formal candidate number, email username and date of birth (the latter often throws up more than one student with the same date, but handwriting can then distinguish). In each of these cases, however, it remains a relatively trivial step for someone with access to the right databases to decode the label and convert it into a name. Of course there is generally no reason why a researcher would want to do this, and students trust that you are not going to waste your precious time doing so.
What else might you do? You could ask the students to pick a bogus name or their favourite superhero, but these run several risks – including having surveys completed multiple “lady gaga”s or “dr [insert your name here]”. The students might also forget the random name they picked between the first and the second test. Continue reading
The Higher Education Academy Centre for Bioscience Pedagogic Research in the Biosciences day conference brought together about thirty academics, for the most part Bioscience specialists, who have been involved to educational research. The day turned out to be highly informative and thought provoking. Some on the hoof reflections were collated via Twitter – click this link.