eProctoring guidance

Introduction

A pilot project between Assessment, UCD Registry and Educational Technology Services, IT Services commenced in the summer 2020 and was completed in April 2022.  The pilot project in UCD is now complete and there is currently no centrally supported eProctoring tool.  While the pilot was undertaken as a response to Covid19 restrictions and took place in the difficult context of the remote teaching and assessment period, a number of key learnings and considerations were highlighted by the Project Team.  These are captured here as guidance for any colleagues looking to procure and use such a tool within a School or College.

Overview

eProctoring is different to other tools that might be used as part of module assessment. There are unique and significant demands on resources involved. There are various tools available but typically eProctoring supports the academic integrity of online examinations through ID verification, reducing access to unauthorised materials and acts as a deterrent to breaches of exam regulations. eProctoring is most effective with assessment of learning type examinations, and less so where assessment as and assessment for teaching models are adopted.  How a system functions varies depending on the solution. Some use AI only, some a combination of AI and a human proctor, and in the case of some Live Proctoring solutions, human proctor only. Privacy and data protection concerns, and possible challenges on those grounds, should not be underestimated and the specific form of eProctoring utilised has a very significant impact on this. eProctoring has proven not to be a panacea that ensures there are no breaches of exam regulations during online exams. Therefore it is critical to consider carefully what tangible value you believe eProctoring can add when planning your assessments.  

Types of Solutions

The following types of solutions are typically available

Important Considerations

• Privacy and Data Protection

Seek advice from the UCD Office of the DPO before procuring a tool for use.  

Data gathered during the eProctoring process should be stored in the EU, or a country with third country status. This is a significant factor as many of the providers store their data in the US or other countries that were not within the EU or considered third countries. There should also be consideration of how University and student data is protected, the significant differences between the various forms of eProctoring, and even between the approaches of different vendors providing the same form of eProctoring. 

• Functionality/Product Specifications

Consider carefully the requirements before deciding on the available functions.

The following are the four key functions of eProctoring solutions and some of the limitations:

• ID Verification: Students can be required to show their ID card to the eProctoring system or proctor via their webcam prior to beginning their exam. If not a human proctor, the system typically takes a photo of the student to validate that the person taking the exam is the person on the ID card.  It may or may not be possible for this to be verified in real-time.
• Prevention: The software can block access to other applications on the device being used for the exam, limiting the opportunity for plagiarism, collaboration, accessing notes or any other unauthorised materials. The effectiveness of this, as a means of preventing misconduct, is limited by the potential use of secondary devices or other external resources. eProctoring cannot prevent use of these resources, only deter students from doing so. 
• Deterrent: Students can be recorded and potential breaches of exam regulations flagged and reviewed. While in most instances, it is difficult to prove categorically that breaches have occurred, eProctoring can act as a deterrent and potentially reduce the instances of misconduct. 
• Gathering Evidence. Where a clearly visible breach has occurred, e.g. where a telephone has been used in view of the camera, or personation occurs, the eProctoring system can provide evidence. In order for data gathered by eProctoring to be taken into consideration as evidence, it must be robust enough to stand up to review and interrogation in a disciplinary or appeal process. Significant resources could be spent on gathering and considering data that would not stand up to review and interrogation. For example, a student may look to a particular location several times in the course of an examination. This may arouse suspicion, and an assumption might be made that collaboration is occuring, or that this student is looking at unauthorised material. The student may, however, simply be considering their answer, or taking a break from the screen for a moment, or at least may explain their actions in this way. To be considered ‘evidence’, the data must clearly show the breach taking place, (e.g. phone clearly being used), not behaviour that may lead one to believe a breach has occurred. While the number of instances of this type of evidence being gathered will be very small, the deterrent value of this function can be significant. Room scans are a particularly invasive aspect of many eProctoring solutions, but in reality do not add significant value. The limitations of room scans have become more widely acknowledged, and in an effort to be more effective, some institutions have adopted multiple camera requirements, or the use of mirrors to provide a view of the entire room. The project team felt that this approach was unacceptably invasive in the context of university exams. Some solutions do not share audio or video data with the Institution as this is regarded as unnecessary data collection. Where evidence is produced by eProctoring, this evidence should be considered as part of the university’s standard protocols and disciplinary procedures. 

• Cost

Consider the requirements in terms of cost and resources

Different costing models will apply to the various approaches, and generally, the costs involved reflect the level of involvement required on the part of the solution provider. Depending on the chosen solution, staff in the institution might need to review flags and footage, which also needs to be considered carefully in terms of data security, time and resources. Support and guidance on procurement requirements can be sought from the Procurement Office.  

Summary of conclusions

• eProctoring can support the academic integrity of online exams, in conjunction with other strategies that form part of assessment design. It is not, however, a ‘silver bullet’ that resolves all concerns regarding the integrity of online exams. 
• The form of eProctoring utilised and the service provider selected have a significant impact on student experience, data protection and privacy, workload implications for faculty/staff and logistical considerations.
• Record and Review solutions seem to provide the best balance between supporting academic integrity and the concerns as noted above.
• Sufficient advance knowledge that eProctoring will be used and clear guidance and information for students is essential for a positive student experience. In particular, reassurance that normal behaviour during exams will not be penalised is important.  
• There are additional workload, resourcing and cost implications associated with the introduction, support and delivery of eProctoring. 
• If dealing with potential breaches, the process should be similar to a plagiarism case, or an exam centre breach, and must align with the Student Discipline Procedure.