An optimum assessment structure will measure student knowledge accurately and without bias. Due to large class sizes (i.e. often 250+ students), first-year biology exams at UCD have for many years been in the form of multiple choice questions (MCQs). Whether or not to use negative marking in these exams has been a matter of great debate, with different decisions being made by the School of Biology & Environmental Science (SBES) and the School of Biomolecular & Biomedical Science (SBBS). Although some studies in educational literature have suggested negative marking leads to bias against female students, such conclusions are not uniform and these studies have mainly been performed at secondary level. Importantly, no studies have been performed investigating first-year biology undergraduate students here in Ireland, or using the exact same negative marking scheme employed here in UCD.

Image: "Immunfluorescent staining of cell membrane transporter proteins in human bladder cells"; an example taken from a lecture slide in one of the stage 1 modules used in the study - BIOL10110 Cell Biology & Genetics.

The overall aim of this study was to determine whether our negatively-marked first-year biology MCQ exams were biased or not against female students. To achieve this, our research team of academic staff from both SBES and SBBS jointly set out to achieve the following goals:

• Determine whether or not there was any bias against female students in the negatively-marked MCQ exams.
• If there was indeed bias observed, to then confirm that removal of the negative marking from the exams did indeed prevent such bias from occurring in those modules. 
• Finally, to put forward a recommendation either way of whether or not negative marking should be used in first-year MCQ exams across the College of Science.

Educational research always faces two key challenges when studying student academic performance:

1. How do you provide a control group for your study WITHOUT detrimental effects to the academic achievement of either the control group or the test group of students? 
2. How do you know that any observed differences are not specific to that student class?

In order to address these two key difficulties, we firstly ran our study over three separate academic years to ensure that there was some consistency across different classes of students. Secondly, for each academic year, we studied the same three first-year biology modules that all contained MCQ exams. The two modules ran by SBES – namely BIOL10110 Cell Biology & Genetics and BIOL10140 Life on Earth – contained negatively-marked MCQ exams and were hence the “test” group. These exams followed standard UCD regulations in that correct answers scored +1, incorrect answers -0.25 and blank answers were given no score. The third module ran by SBBS – namely BMOL10030 Biomedical Sciences – contained no negative marking and was hence the “control” group. Crucially, we only included students in our analysis that (a) had taken all three modules, and (b) completed, at least in part, every assessment component of each module.

NOTE: Ethical approval for this study was obtained from the University College Dublin Research Ethics Committee (Reference number LS-E-15-102-Stewart).

So far, in both of the first two full years of this study, the academic performance of a large number of students has been analyzed, with 188 students in one year (121 female; 67 male) and 151 students in the next year (97 female, 54 male). For each year, no gender differences in academic performance were observed in the BMOL10030 MCQ exam which did not contain negative marking. In contrast, in both years a statistically significant bias against female students was consistently observed in the BIOL10110 and BIOL10140 modules in which negative marking was applied (P<0.05, t-test), with female students scoring on average between 5 and 8% less in the MCQ exam. Intriguingly, further analysis appeared to confirm that this difference was at least partly explained by a cohort of female students displaying “risk averse behaviour”, in which they passed on a large number of questions. This interpretation was further strengthened by the fact that there were no gender-based differences in marks obtained for other assessments in these two modules, such as practical class write-ups and on-line continuous assessment (NS, t-test).

In June 2017, these data were presented to all the SBES academic staff at their annual Teaching & Learning Day. The decision was then made to remove negative-marking from all SBES MCQ exams for the 2017-18 academic year and to carefully monitor whether this action did indeed remove the bias previously observed against female students. Although not yet complete, preliminary analysis of the semester 1 BIOL10140 module ran in late-2017 shows that the removal of negative marking did indeed completely remove the bias as predicted. Importantly, there was less than 1% between male and female MCQ exam academic performance. If this pattern is repeated when the full analysis for 2017-18 is complete, we will then recommend that (i) SBES continues with its new policy of not using negative marking, and (ii) that the College of Science (and maybe other UCD colleges) should strongly consider doing the same, if they are not already doing so.