Exam Proctoring Technologies

While exam proctoring technologies have been around since 2008, they have reached peak popularity during the COVID-19 Pandemic. The systems provided a solution many schools, including high schools and universities, needed to deter cheating in their online testing environments. Exam proctoring technologies are computer tracking systems for schools, universities, and other organizations to monitor a remote student's computer and ensure that they are using the appropriate resources during an exam . These technologies typically demand access to a student’s desktop, camera, and microphone. Some of the technologies also track a student’s eye movements during the exam, making sure they are only looking at the screen. Several ethical concerns have been raised in regards to this technology such as privacy and accessibility.

Perspectives

User (Student) Experience of Proctoring Technologies

Before an exam, the student must log in to a website or download an app on their device to access the testing environment. Upon joining the application or website, the student will be notified that their webcam and microphone are being monitored . Then, the student will be prompted to take a picture of their face, an identification card, and any other information relevant to the exam . During the exam, the student will see the questions as normal as the proctoring software monitors their device. If the exam requires it, a “lockdown browser” feature will also be enabled. This prevents the student from accessing the internet or any other applications on their device . After the student has taken the exam, they will be notified by the school or organization whether they were caught violating the rules of the exam. The penalty for cheating varies depending on the specific situation. Common penalties are receiving a failing grade on the exam or course, severe point deductions, or revoking an award or certification .

University and Organization Perspective

Online proctoring technologies are attractive for organizations and schools because they can provide environments similar to taking exams in person with a human proctor in the room. Within this context, the organizations aim to “police the integrity” of the exams and verify that the results appropriately convey the student’s understanding of the material . The cost is also an advantage to these organizations. Instead of setting up a testing site and hiring human proctors, organizations can simply hire an online proctoring service to handle testing integrity . During the exam, instructors and organizations are responsible for differentiating between academic dishonesty and fair test takers . Depending on the system used, an instructor's role can vary from directly viewing the student for the entirety of an exam or reviewing alerts raised by artificial intelligence . At the end of the exam, it is the organization's responsibility to make determinations of what actions are deemed academic dishonesty. They do this by reviewing the video recordings that were flagged during the exam. The artificial intelligence systems simply make recommendations of what actions could be considered suspicious and the organizations review them .

Instructor Perspective

Instructors have mixed perceptions of the use of online supervision technologies in their classrooms. In a study garnering students’ and instructors’ perspectives on online exams, instructors conveyed that they thought the technologies helped ensure fair test taking. Instructors who did not share this viewpoint mentioned that switching their exams to an open book format was more effective for their students. These instructors also mentioned how it was much more convenient to provide feedback with the online exam supervision formats as opposed to in person formats .

Proctoring Technology Companies

The most popular online exam proctoring companies emphasize how their technology can benefit students and instructors alike. Honorlock, one of the leaders in the space , emphasizes the flexibility of their technology. In particular, they advertise that instructors reap benefits as their technology can adapt to serve different testing formats. Along with testing formats, the technology provides flexibility to students as it has an option to allow students to schedule their own exams. The technology still monitors them, but the instructor can review flags raised by the AI at a later time . Proctor Track, another leader in the space, emphasizes the time saved by having online proctoring. Typically, students and instructors commute to a testing location, which is eliminated with the use of this technology. They also state that organizations can have a larger reach and more students under their services. While reaching a larger audience, Proctor Track ensures that their technology makes exam results credible and deters academic dishonesty with the use of their AI .

Types of Proctoring

Human Proctored Exams

Human proctored exams involve a human proctor viewing a test taker for the entirety of the exam to detect violations of exam integrity in real time. If a violation is seen, the human proctor interferes immediately, dismisses the student, and voids the exam grade. This method was introduced in 2006, but gained relevance in 2015. Despite the rise in popularity, this methodology has had disadvantages. Proctors and instructors noted that it was difficult for them to monitor students' webcam and desktop at the same time. This combined with the fact that many proctors are monitoring multiple students made this monitoring technique less reliable . In light of these disadvantages, human proctoring has seen a drop in popularity after 2015 and hybrid solutions using artificial intelligence combined with human proctoring have become increasingly popular .

Data Collection

There are several different types of proctoring technologies and companies that collect a wide variety of different data. The most common data collected is a user’s webcam, keyboard strokes, microphone, and browser information . Often, this data is stored in a database for a given amount of time until it is deleted. Most exam proctoring companies such as ProctorU state that they will not store data for longer than a year before it is deleted . These basic features are just the tip of the iceberg in terms of the technology used to track students. Students can also be asked to give a 360 degree view of their working environment prior to the start of the exam. These checks are known as “room scans” .

Artificial Intelligence

Artificial intelligence algorithms are also being used to further collect data on students. These algorithms can be used to verify the identity of students taking an exam. Facial detection scans are one of the applications of these algorithms where students are required to participate in the scan before an exam . Other verification techniques include presenting an identification card or government document through the camera . The most extensive use of artificial intelligence is done during the exam by tracking eye and head movements to determine whether students are cheating off screen. With machine learning, developers can train a model with thousands of examples of people who are taking the exam fairly and cheaters. This model can then be used in practice to detect suspicious head and eye movements typically seen in cheaters. When suspicious activity is detected, a human reviews the footage and makes a final determination on whether the student was cheating or not .

North America

The adoption of exam proctoring technology has seen mixed results in the United States and Canada. The United States shows a high adoption of the technology with near 66% of universities mentioning some form of online proctoring. This is much higher than Canada’s adoption of these technologies as they stand at just 39.2% . Despite the acceptance of the technologies among universities, student petitions and complaints continue to pile against the implementation of online proctoring . In the United States, lawsuits have been filed against Proctorio from students and privacy groups alike. One of the biggest of these is a complaint brought up by The Electronic Privacy Information Center (EPIC) against five of the top exam proctoring services. EPIC claims that these companies have not been transparent with the data they are collecting from students and how it is being obtained. They also complain that the data collected by these companies is “excessive and harmful.” EPIC is concerned about how collecting this data can also cause anxiety among students and harm the equity of exams . In Canada, a litigation case took place between Ian Linkletter and Proctorio. Linkletter is an education technology specialist who has tweeted YouTube videos from Proctorio’s account showing their technology from the perspective of an instructor. It showed how instructors can see student eye movement and other information. After taking down their YouTube videos, Proctorio sued Linkletter for copyright infringement and “distributing confidential information.” The law sided with Proctorio and shut Linkletter down. Along with critics of exam proctoring technology in Canada, the University of Ottawa made a report stating that these technologies do not meet privacy requirements. The study points out an example of how the noises of a “domestic pet” in a student’s household have been flagged. The study concludes that the ultra sensitive technology can cause students to be falsely accused of cheating if they live in a house with more noise. This study was supported by the Office of the Privacy Commissioner of Canada .

Europe

Students in the EU first expressed their concerns over privacy of online supervision technologies in 2020. During this time, the student council at The Amsterdam University is attempting to ban the use of Proctorio within their exams. The students argued that law in the EU requires “explicit” consent for services to access their data . While students agreed to Proctorio’s terms before taking exams, they argued that there was no alternative to taking tests and they were pressured into giving up their data. Despite the students’ concern, courts in Amsterdam sided with Proctorio and agreed that they were properly handling student data . The London School of Economics shares a view siding with students and has not taken on this software .

Ethical Concerns

Academic Dishonesty

Academic dishonesty is when a student aims to gain an unfair advantage in “formal education exercises.” This is particularly prevalent with online examinations as students do not have direct, in person supervision from instructors. There is ongoing debate, in the context of online exams, between finding the balance of preventing academic dishonesty while also respecting students’ privacy. According to a study done by Lori Mcnab in the Merlot Journal of Online Learning and Teaching, both students and faculty perceived that it is easier to cheat online than in person . Another study compared online exam scores of when students were supervised and unsupervised. The study, done by Seife Dendir, concluded that online supervision technologies do deter cases of academic dishonesty . Further, Dendir noted that live proctoring does not do enough to deter dishonesty and storing recordings of student webcams is the ideal method. On the contrary, other views attribute differences in exam scores to the stress and anxiety that online supervision technologies have caused students. Students have increasingly viewed the technology as “intrusive” and “uncomfortable”. Gabi Martorell, a psychology professor at Virginia Wesleyan University, shares these views and has shifted the methodology of her classroom to specific open book assignments. These assignments are harder to find online and don’t require supervision technology .

Privacy

Perhaps one of the most important and relevant concerns for exam proctoring technologies is privacy. During the COVID-19 Pandemic, students typically took exams from inside their homes as access to libraries and other areas was restricted. Due to this, “room scans” were being recorded inside students’ homes often in their bedrooms. According to NPR, a student at the Cleveland State University was in a similar situation and sued his school after he had taken the scan. The student, Aaron Ogletree, claimed that the scan was an “unreasonable search and seizure” of his home and the court ultimately sided with him. The defense claimed that Ogletree had control of what he displayed in the room and could have refused to do the scan, giving him a zero on the exam. The judge, however, ruled that these searches “go where people otherwise would not” and that the school would not have access to Ogletree’s room in the first place without the proctoring system .

Additionally, exam proctoring companies hold sensitive information such as video, audio, and personal information in their databases, which may be vulnerable to attack. In 2020, ProctorU, one of the leaders in exam surveillance technology, suffered a data breach where 444,000 records were leaked. These records included names, emails, phone numbers, passwords, addresses, and affiliated organizations .

Discrimination

Exam proctoring algorithms have been shown to be unable to identify students with darker skin tones. This puts them at a disadvantage in stressful testing environments because they spend a large amount of time getting the technology to work instead of working on the exam. The experience of Ameya Ross, an African American student at Ohio State University, is one example of these issues at play. According to The Guardian, Ross took precautions prior to the exam to ensure she was in a well lit area. Even after taking these precautions beforehand, it took her 45 minutes to properly scan her face while students of lighter skin tones could begin immediately. The system finally recognized her face after she used a bright light directly over her head, which was a very uncomfortable environment . Ameya’s situation is not uncommon. Lucy Satheesan, a software researcher, has done studies on facial recognition of Proctorio, a popular online proctoring company, and claims that the technology fails to identify black faces over 50% of the time . Satheesan’s study included over 11,000 faces of different races and ethnicities. Her raw data showed only a 42.99% pass rate for black people and only a 59.39% pass rate for Middle Eastern people .

Accessibility

In order to implement online proctoring for students, providing a system everyone can access is vital. There are many different issues to consider including students with unstable homes, no access to internet connection, no access to a device, and more. Often, institutions work with software proctoring companies to help provide access and services for students when needed. They provide extra time, extra breaks, speech to text options, magnifying glasses, human assistance during exams, device access, and more to students who need it. . Even with a wide array of services some students still faced a hard time adjusting to online exams. Internet inaccessibility is one of the most difficult problems to solve, especially during the pandemic. Students without access to consistent, high speed internet face more interruptions and disadvantages while taking exams. This inequality directly affects a student’s success (their grade point average) and confidence in navigating online assessments .