VOL. 34, No. 1 2019
Abstract: It is safe to say that online leaning has found a permanent place in higher education. Conventional higher education institutions are also gradually embracing it across the United States. As online learning surfaces as the new model of contemporary education both in the United States and worldwide, ensuring exam integrity in the online environment is becoming a major challenge to many higher education institutions. To meet this challenge, many of these institutions are outsourcing the examination aspect of their education to online proctoring service providers. The present study, which was conducted on a total of 631 students, assesses the effect of online proctored exams on student test anxiety and exam performance. This study shows that high trait test anxiety results in lower exam scores and that this is especially true for those students with high text anxiety taking exams in an online proctored setting.
Keywords: online proctoring, learning, test anxiety, worry, emotionality.
Résumé : On peut affirmer sans risque que l’apprentissage en ligne a trouvé une place permanente dans l’enseignement supérieur. Les établissements d’enseignement supérieur conventionnels l’adoptent également progressivement aux États-Unis. Alors que l’apprentissage en ligne apparaît comme le nouveau modèle de l’éducation contemporaine tant aux États-Unis que dans le monde entier, assurer l’intégrité des examens dans l’environnement en ligne devient un défi majeur pour de nombreux établissements d’enseignement supérieur. Pour relever ce défi, bon nombre de ces établissements confient l’aspect examen de leur formation à des fournisseurs de services de surveillance en ligne. La présente étude, qui a été menée sur un total de 631 étudiants, évalue l’effet des examens en ligne surveillés sur l’anxiété des étudiants et leur performance aux examens. Cette étude montre que lorsque le niveau d’anxiété détecté par le test est élevé, les résultats d’examen sont plus faibles et que cela est encore plus fort pour les étudiants dont l’anxiété est élevée qui passent des examens dans un contexte de surveillance en ligne.
Mots-clés : surveillance en ligne, apprentissage, test d’anxiété, inquiétude, émotivité
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
Online learning is an integral part of contemporary education in the United States, and globally. A 2011 study conducted by Ambient Insight Research indicates that over 1.25 million students took all their courses online (Adkins, 2011). The 2018 Babson Survey Research Group report shows that between 2015 and 2016 over 30% of post-secondary students—or 6.5 million—took at least one course online; the majority of these students, almost 70%, were from public institutions (Seaman, Allen & Seaman, 2018). Likewise, a 2018 Canadian study based on over 200 higher education institutions, indicated a significant growth in online learning, with those institutions increasing their online offerings by 11% (Bates, 2018). Indeed, Bhagat, Wu and Chang (2016) reported that online learning integrated with social network connectivity is providing students and educators with an ecosystem of interaction and troves of learning resources. This is in part because online learning allows students the convenience and flexibility to better fit their studies in with their work and other obligations and, as Mann and Henneberry (2012) show, many of those enrolled in online courses are of typical college age, i.e., 18-24.
Although the role of technology in the teaching and learning environment is well established and goes back centuries, the recent rapid development of information and communication technologies has made the role of educational technology in teaching even more important (Stošić, 2015). In a 2013 study Harden asserted that given our advancements in information technology, the college classroom itself has in part become virtual. Certainly, technology can significantly improve the learning experience of students – bringing concepts and curricula to life in new ways (Woldeab, Lindsay & Brothen, 2017).
A number of scholars (e.g., Carey, 2016; DeMillo, 2011) have stressed that the centuries-old models upon which US higher education is based is inadequate for the twenty-first century, and that information technology is capable of providing quality and affordable forms of higher education. However, this shift has its own challenges. While it might be true that the new paradigm gives conventional higher education institutions an opportunity for growth and the ability to expand their course offerings, it is also posing a challenge in how to best ensure exam integrity. Therefore, this study aims to fill the following research gap: while online proctoring may provide a solution to the question of exam integrity, the negative effects of such an intrusive type of monitoring on students and exam performance are not yet well known.
As conventional higher education institutions broaden their online offerings, more and more students are taking their exams online (Stowell & Bennett, 2010). This trend is not limited to online offerings either. As far back as 2001 Alexander, Bartlett, Truell, and Ouwenga noted that many educators in traditional face-to-face courses were also moving their exams online, among others, to save time grading and to free up class time. This trend pointed to the need for maintaining academic integrity through proctored online assessments.
Hylton, Levy and Dringus (2016) state that “deception and dishonesty in online exams are believed to link to their unmonitored nature where users appear to have the opportunity to collaborate or utilize unauthorized resources during these assessments” (p. 53). Likewise, Faucher and Caves (2009) stressed that academic dishonesty most often happens when students have opportunities to cheat because of reduced surveillance. Indeed, the Witherspoon, Maldonado and Lacey (2012) study showed that nearly 80% of those surveyed were involved in academic misconduct; the same researchers noted that those involved indicated that they would engage in misconduct if the opportunity arose. On the other hand, Karim, Kaminsky and Behrend (2014) concluded their review by warning of the unintended negative effects on test takers when remote proctoring is used to reduce cheating.
As every aspect of education requires some kind of assessment, students throughout their education journey are expected to have their learning measured in a variety of ways. The results are used not only to gauge their gains in knowledge but also the effectiveness of the instruction itself and the overall viability of programs and curricula (Salend, 2009). However, these assessments can also subject students to test anxiety, which can negatively impact their performance (Huberty, 2009).
Studies on test anxiety go back to the early 20th century, with the first one published in 1914 by Folin, Denis and Smillie. Almost four decades later many more studies emerged on test anxiety and its connection to performance (e.g., Liebert & Morris, 1967; Morris, Davis & Hutchings, 1981; Sarason, Mandler, & Craighill, 1952) leading to the present distinction between state and trait anxiety. For example, Mandler and Sarason (1952) used responses on a test anxiety survey to characterize research participants as having either high or low test anxiety, and found that students with low test anxiety did much better on intelligence tests than students with high test anxiety.
Early studies in this area theorized test anxiety to be one-dimensional (e.g., Sarason, 1961). However, that understanding shifted in the late 1960s, as state test anxiety was understood to contain elements of worry and emotionality. Indeed, Hong and Karstensson (2002) noted that “state test anxiety refers to the transitory, anxious affect state provoked by a specific evaluative situation, while trait test anxiety refers to the tendency to be anxious in any evaluative situation” (p. 349). Contrary, trait test anxiety can differ in its intensity, extent, and the range of circumstances in which it happens and consequently is the more general condition. Individuals with test anxiety are typically considered to have the more narrowly-defined state test anxiety; this means that their elevated levels of stress are situation specific, as they are the result of certain evaluative activities, or conditions, such as difficulty of the material or the student’s lack of preparedness (Cassady, 2009). On the other hand, researchers have found that students who display greater trait anxiety experience higher state test anxiety than students with lower trait anxiety (Head, Engley & Knight, 1991). In this research study, because of the difficulty of assessing students’ individual situations, we used a more general trait measure to assess students’ test anxiety and then determine its relationship to test performance.
In an educational assessment setting, test anxiety may often distort and disguise the true abilities of students (Meijer, 2001). However, the understanding around online proctored examinations and test anxiety is very limited. In fact, our literature search looking into proctored online exams and test anxiety was unfruitful. The few studies in this area have to do with unproctored exams (e.g., Stowell & Bennett, 2010), or with online exams defined as students testing in a secure computer laboratory (e.g., Cassady & Gridley, 2005) and suggest that online exams are better for students with high test anxiety.
For example, the Stowell and Bennett (2010) study looking into the use of course management software to conduct face-to-face class exams online (i.e., unproctored online exams) found that students who typically show high levels of test anxiety in a classroom had less anxiety when taking their exams online, while the opposite was true for students showing low classroom anxiety. Similarly, the study of Cassady and Gridley (2005), which compared students who took their exams using paper and pencil and those online (i.e., in the computer laboratory), found that students who took their exams online reported lower levels of perceived test anxiety.
Conversely, the present research study builds on a preliminary study conducted by Woldeab and others in 2017, which suggested that one of the main concerns students have about online proctoring is proctor intrusiveness. More specifically, the purpose of this current study is to assess if actual online proctored exams induce higher levels of test anxiety and result in lower student performance.
This study looked at 631 undergraduate students attending a large midwestern US university. The data was collected from a large introductory psychology course with three types of sections: face-to-face, online, and hybrid. Data collection took place during the fall and following spring semesters. These students completed all the relevant measures we used in this study and also gave their consent for us to use their data. Of the total of 631, 44 students took their final exam via ProctorU (PU group) and served as the “experimental” group. The remaining 587 took their exams in the computerized testing center (TC group) and thus served as the “control” group. We therefore report our findings derived from both the experimental and the control groups, who took both the pre- and the post-surveys.
Students taking their exams with ProctorU, an online proctoring service provider, took their exams individually by signing up online and coordinating their exam time with ProctorU staff. They were monitored by ProctorU staff via a webcam. Those who took their exams through the proctored computer testing center did so in the presence of their peers and were monitored by the testing center staff.
Participation in this study was entirely voluntary. Research participants were provided a consent form, which gave specific information about the study, including the purpose as well as the risks and benefits of their participation. Research participants were asked to electronically sign the consent form both before the pre-survey and before the post-survey. To encourage participation in the study, the faculty of the participating courses agreed to give two extra course points to those surveyed.
To assess reactions to proctored online examinations, participants were given pre- and post-surveys that measured their overall expectations about and reactions to proctored online examinations. For this study, we examined student responses to the post-survey item asking them to rate – on a scale of comfortable to uncomfortable – their experience of being monitored by a proctor.
To measure participants’ trait anxiety, we used the Westside Test Anxiety Scale developed by Driscoll (2007). The 10-item scale was developed to assess performance impairments. Most of the questions in this scale directly address performance impairments related to cognitive symptoms of anxiety, i.e., lack of attentiveness, poor memory, or worry. According to Driscoll (2007), the scale consists of six items assessing impairment, and four items assessing worry and dread. The questionnaire includes no items on physiological over-arousal. In fact, the “Westside scale thus has high face validity, in that it includes the highly relevant cognitive and impairment factors but omits the marginally relevant over-arousal factor” (Driscoll, 2007, p. 2). Numerous studies have shown repeatedly that lower scores in the Westside scale are related with better test performance, which shows the scale to be a reliable and valid measure (Driscoll, 2007).
To ensure we were measuring participants’ trait anxiety rather than anxiety about an impending exam, the Westside measure was administered in the pre-survey during the first week of the semester. Also for this study, we focused on the final survey questionnaire item that asked students whether and to what extent the ProctorU proctors made them uncomfortable. Finally, we used participants’ ACT scores, GPAs, and cumulative final grade as control measures.
In this study, we set out to explore online proctoring and test anxiety, specifically whether: a) high test anxiety students report difficulties with online proctoring, and b) online proctored exams induce higher levels of test anxiety and result in lower student performance. A total of 631 students completed all the relevant measures we used in this study and also gave their consent for us to use the resulting data.
As noted above, a total of 44 of these students took their final exam monitored by ProctorU and served as the “experimental” (PU) group. The remaining 587 took their exams in the computerized testing center and thus served as the “control” (TC) group. Because we did not randomly assign students to experimental and control groups, we assessed their scores on five variables relevant to our study’s purpose that would indicate if the groups were comparable: the Westside Anxiety scale; final exam performance; ACT scores; GPA; and total credits completed (see Table 1).
Table 1: Group Means
Group |
n |
GPA |
ACT |
Comp Cr |
Anxiety |
Exam |
Testing Ctr |
587 |
3.35 |
27.39 |
48.46 |
28.33 |
156 |
ProctorU |
44 |
3.30 |
26.16 |
59.98 |
28.48 |
155 |
First, we examined students’ scores on the Westside Anxiety scale. Both groups had nearly identical scores—differing by about two hundredths of a standard deviation and obviously not statistically different from each other. Second, final exam performance between the two groups differed by only one point out of a possible 200 (155 vs 156) and was also obviously non-significant. Thus, the two central variables of interest in this study did not differ overall between the two groups. As to our concern for getting a clear picture of the effect of online proctoring on exam performance, we found no indication that trait test anxiety and academic ability differed between our two comparison groups and therefore skewed the results.
We next examined students’ scores on the Comprehensive portion of the American College Testing college readiness assessment (ACT), their cumulative grade point averages (GPA), and their cumulative college credits completed (CCR), all obtained from the registrar’s office. The GPAs of the two groups also did not differ practically or statistically (3.30 vs 3.35). However, the groups did differ on the two other variables. TC students had higher ACT scores (M = 27.39, SD = 3.15) than PU students (M = 26.16, SD = 3.61), and this difference was statistically significant by t-test (t = 2.09,
p = .037). Conversely, PU students had completed more credits (M = 59.98, SD = 40.94) than TC students (M = 48.46, SD = 32.72) and this difference was also significant (t = 2.22, p = .027). For the purposes of this study, we argue that these differences balance out somewhat because academic ability can be assessed with either of these measures. That interpretation is supported by the finding reported above that final exam performance between the two groups did not differ. Together, these findings provide support to the contention that the PU and TC groups were similar in important ways so that the different exam environments can be assessed accurately.
Overall, we found a relationship between trait test anxiety as measured by the Westside scale, and students’ exam performance (r = -.167 p = .000); not surprisingly, we found higher anxiety was associated with lower exam scores, just as decades of research on test anxiety has found. More importantly, this relationship was stronger for PU students (r = -.443, p = .003) and we found in a test for differences between correlations that this difference was marginally significant (z = .47, p = .06).
In a closer examination of our data, we found that the greater relationship for trait test anxiety and poorer final exam performance among the PU students was mostly restricted to those with high anxiety scores. High anxiety PU students (scores at the median score of 29 and higher) performed less well on the exam than those below the median (High anxiety M = 144.62, SD = 24.83; Low anxiety M = 162.25, SD = 24.63; t = 2.52, p = .015). Finally, the high anxiety PU students’ reactions to the online proctors suggest why this was the case. Students reported that proctors made them uncomfortable (i.e., had lower scores on the comfort scale) more so than those with a mean anxiety score below 29 (High anxiety M = 2.09, SD = .868; Low anxiety M = 2.67, SD = 1.02; t = 2.00, p = .052). Thus, we found strong indications that online proctoring had a negative effect on students with high anxiety.
Our review of the literature did not reveal a significant body of previous studies in which to situate our findings. As demonstrated by our review, there are very few studies in the realm of online proctoring specifically relating to the nexus of test anxiety and exam performance. The few studies we came across do not deal with online proctoring (i.e., webcam monitored examinations—especially those using live proctors). Online proctoring is relatively new and systematic research has not yet caught up with this technology. We therefore assert that more research in this area is needed—some of which we report here—and draw three conclusions from our review of the literature and our data. First, it seems clear that online learning will continue to grow and that this will increase the need for online testing. Second, we found little prior research on how the significant move to online testing in higher education affects different types of students. Third, the results from this study support the findings of a preliminary study conducted by Woldeab and others (2017), which indicated that an area needing further research is online test anxiety. Consistent with previous research showing that high trait anxiety interferes with exam performance, students in this study had lower scores if their anxiety levels were high, but this effect was greater for those students in the online proctoring group who reported high trait test anxiety.
In the research reported here, we have identified an issue that seems to have escaped the attention of researchers studying online learning—test anxiety. The particular effect we found combines what may be a general wariness of technology with some students’ fear of testing. The fact that students showed practically no difference in their final exam scores, when comparing online proctoring with in-person testing, indicates that the negative effects of online proctoring may generally be hidden. But these negative effects appeared when we examined the scores of students with high trait test anxiety. Further inspection found this online proctor effect—that having a live proctor watching them was most upsetting to the high anxiety students and that it interfered with their exam performance—to be in line with a preliminary study conducted by Woldeab and others (2017). We found evidence for a person-situation interaction—that test anxiety interacted with the testing situation and resulted in those students who were monitored by an online proctor obtaining lower scores if their anxiety was high. This interaction resulted in some students being disadvantaged by a common feature of online test monitoring services.
The findings of this study should pave the way for further exploration in this area and help us expand our understanding of how online proctoring affects the important relationship of student anxiety to exam performance. Certainly, further research in this area would help more students to be successful. However, perhaps more important is that we have identified a need that college and university counseling centers should address in their programming, and that instructors should consider when using online proctoring: the general wariness of technology combined with students’ fear of testing makes online proctored exams very difficult for some students, in particular those who already exhibit trait anxiety.
Adkins, S. S. (2011). The US market for self-paced eLearning products and services: 2010-2015 forecast and analysis. Ambient Insight.
Alexander, M. W., Bartlett, J. E., Truell, A. D., & Ouwenga, K. (2001). Testing in a computer technology course: An investigation of equivalency in performance between online and paper and pencil methods. Journal of Career and Technical Education, 18, 69-80.
Bates, T. (2018). The 2017 national survey of online learning in Canadian post-secondary education: Methodology and results. International Journal of Educational Technology in Higher Education, 15(1), 29.
Bhagat, K. K., Wu, L. Y., & Chang, C. Y. (2016). Development and Validation of the Perception of Students Towards Online Learning (POSTOL). Educational Technology & Society, 19(1), 350-359.
Carey, K. (2016). The end of college: Creating the future of learning and the university of everywhere. Riverhead Books.
Cassady, J. C. (2009). Test anxiety: Contemporary theories and implications. In J. C. Cassady (Ed.), Anxiety in schools (pp. 7–26). New York: Peter Lang.
Cassady, J. C., & Gridley, B. E. (2005). The effects of online formative and summative assessment on test anxiety and performance. Journal of Technology, Learning and Assessment, 4(1) 1-31.
DeMillo, R. A. (2011). Abelard to Apple: The fate of American colleges and universities. MIT Press.
Driscoll, R. (2007). Westside Test Anxiety Scale Validation. ERIC: Online Submission. https://eric.ed.gov/?id=ED495968
Faucher, D., & Caves, S. (2009). Academic dishonesty: Innovative cheating techniques and the detection and prevention of them. Teaching and Learning in Nursing, 4(2), 37-41.
Folin, O., Denis, W., & Smillie, W. G. (1914). Some observations on "emotional glycosuria" in man. Journal of Biological Chemistry, 17(4), 519-520.
Harden, N. (2013). The end of the university as we know it. The American Interest, 8(3), 54-62.
Head, L. Q., Engley, E., & Knight, C. B. (1991). The effects of trait anxiety on state anxiety and perception of test difficulty for undergraduates administered high and low difficulty tests. Journal of Instructional Psychology, 18(1), 65-68.
Hong, E., & Karstensson, L. (2002). Antecedents of state test anxiety. Contemporary Educational Psychology, 27(2), 348-367.
Huberty, T. J. (2009). Test and performance anxiety. Principal Leadership, 10(1), 12-16.
Hylton, K., Levy, Y., & Dringus, L. P. (2016). Utilizing webcam-based proctoring to deter misconduct in online exams. Computers & Education, 92, 53-63.
Karim, M. N., Kaminsky, S. E., & Behrend, T. S. (2014). Cheating, reactions, and performance in remotely proctored testing: An exploratory experimental study. Journal of Business and Psychology, 29(4), 555-572.
Liebert, R. M., & Morris, L. W. (1967). Cognitive and emotional components of test anxiety: A distinction and some initial data. Psychological reports, 20(3), 975-978.
Mandler, G., & Sarason, S. B. (1952). A study of anxiety and learning. The Journal of Abnormal and Social Psychology, 47(2), 166.
Mann, J. T., & Henneberry, S. R. (2012). What Characteristics of College Students Influence Their Decisions to Select Online Courses. Online Journal of Distance Learning Administration, 15(4), 1-14.
Meijer, J. (2001). Learning potential and anxious tendency: Test anxiety as a bias factor in educational testing. Anxiety, stress and Coping, 14(3), 337-362.
Morris, L. W., Davis, M. A., & Hutchings, C. H. (1981). Cognitive and emotional components of anxiety: Literature review and a revised worry-emotionality scale. Journal of Educational Psychology, 73, 541-555.
Salend, S. J. (2009). Classroom testing and assessment for all: Beyond standardization. Thousand Oaks, CA: Corwin Press.
Sarason, S. B., Mandler, G., & Craighill, P. G. (1952). The effect of differential instructions on anxiety and learning. The Journal of Abnormal and Social Psychology, 47(2), 561-565.
Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade Increase: Tracking Distance Education in the United States. Higher Education Reports: Babson Survey Research Group.
Stošić, L. (2015). The importance of educational technology in teaching. International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 3(1), 111-114.
Stowell, J. R., & Bennett, D. (2010). Effects of online testing on student exam performance and test anxiety. Journal of Educational Computing Research, 42(2), 161-171.
Witherspoon, M., Maldonado, N., & Lacey, C. H. (2012). Undergraduates and academic dishonesty. International Journal of Business and Social Science, 3(1), 76-86.
Woldeab, D., Lindsay, T., & Brothen, T. (2017). Under the watchful eye of online proctoring. Innovative learning and teaching: Experiments across the disciplines. Minneapolis, MN: University of Minnesota Libraries Publishing.
Authors
Daniel Woldeab is an assistant professor in the College of Individualized Studies at Metropolitan State University. He holds a bachelor’s degree in computer information systems, a master’s degree in education and a doctoral degree in Work and Human Resource Education (adult education). His research interests include online education, adult literacy and adult education, technology and pedagogy, technology and culture, cultural competency, acculturation, and acculturative stress and anxiety. E-mail: daniel.woldeab@metrostate.und.edu
Thomas Brothen is Morse-Alumni Distinguished Teaching Professor in the Department of Psychology at the University of Minnesota-TC and holds bachelor’s and PhD degrees in psychology. His primary research has involved developing and examining online course management systems and other technology to improve post-secondary student learning; the teaching of psychology and how technology can be utilized to improve it; and the use of psychological theory to guide large-scale educational interventions. Email: broth001@umn.edu. Email: broth001@umn.edu