Top marks for ChatGPT in the Leaving Certificate Computer Science Examination
ChatGPT Plus has demonstrated the ability to successfully pass the Higher Level (opens in a new window)Leaving Certificate Computer Science (LCCS) examination by answering nearly all questions correctly.
The LCCS examination consists of an end-of-course examination worth 70% of the grade and coursework assessment worth 30%. We conducted an experiment that utilized the 2021 LCCS Higher-Level examination, which included 16 questions with a total of 62 sub-questions. The sub-questions were presented to ChatGPT in separate chat sessions without any additional information. The answers provided by ChatGPT were then compared to the official Leaving Certificate marking scheme, resulting in a majority of correct answers (55 out of 62).
There is an element of choice in the LCCS examination, so considering only ChatGPT's best answers would likely yield close to or possibly 100%, depending on the marker's interpretation of the marking criteria. This would correspond to the highest grade possible (H1) in the 2021 Higher Level exam. With the examination comprising 70% of the overall grade, achieving a grade of H3 would be possible even before considering the coursework component worth 30%. For comparison, only (opens in a new window)14% of students achieved a H1 in 2021.
The Leaving Certificate is an invigilated exam, and was invigilated long before Generative AI, and for good reason. So although ChatGPT can excel in the LCCS, it does not compromise the exam's integrity. Furthermore, the LCCS includes a (opens in a new window)coursework assessment component, and this aspect is supervised by the students' teachers. While ChatGPT could be used for this, the nature of this project work is much more involved than the questions found on a paper exam. You can read more about this work at (opens in a new window)brettbecker.com or at (opens in a new window)independent.ie.
Joyce Mahon is a Computer Science PhD student at UCD and part of the Science Foundation Ireland Centre for Research Training in Machine Learning ((opens in a new window)https://www.ml-labs.ie/ ) supervised by (opens in a new window)Dr. Brett Becker and (opens in a new window)Dr. Brian Mac Namee.
Published 04.07.23