As about 30,000 would-be lawyers prepare to take online bar exams in New York, California, and 16 other states in early October, many have said they’re grateful to avoid the health risks that come with taking an in-person test alongside hundreds of others during a pandemic.
Yet the online tests also present a range of significant issues, adding another layer of uncertainty for those preparing for the biggest test of their professional lives.
There are fears over the security and effectiveness of facial recognition technologies and artificial intelligence-driven camera watchers, or “remote proctors,” meant to combat cheating. There are also concerns about whether certain test takers, including pregnant women and those with disabilities or medical conditions will get a fair shake.
On a recent Reddit string, several pregnant women planning on taking the online tests in different states expressed concerns about requirements that they sit in full camera view, except for scheduled breaks, to avoid cheating. They said their physical conditions make it difficult to sit without a bathroom break for the 90 minute stretches administrators are demanding.
“Iffy on a good day but since I’ll be in my third trimester of pregnancy, it is down right impossible,” wrote Kristin Leuchtag, who graduated from the University of Akron School of Law in 2019, and told Bloomberg Law she plans to take the Ohio bar exam. “Anyone who’s ever been pregnant or spent significant time around a pregnant woman knows that this is ridiculous.”
Leuchtag said in an interview that she’s asked for accommodations but hasn’t yet heard back from Ohio bar administrators. Representatives of the Ohio Supreme Court and its Office of Bar Admissions did not respond to emailed questions. Several other states’ bar administrators, including New York’s, declined to comment.
And none of this is to mention the risk of technical glitches or breakdowns that have occurred in each of the individual states that have attempted to hold an online bar to date. Run-up tests designed to make sure the full scheduled exams would actually work revealed system flaws that caused states like Florida to delay their exams, or to move to “open book” tests that aren’t reliant on vendor proctoring and software.
Mounting concerns have only stoked the movement for alternatives to the bar exam, at least for the remainder of 2020. One option is a “diploma privilege,” which allows states to license those who don’t take the bar.
‘Are You Kidding Me?’
Each state since July that has attempted to hold an online exam with the aid of tech vendors ExamSoft and ILG Technologies—including most recently, Florida—has run into glitches.
Florida’s online exam scheduled for Aug. 19 was canceled less than 72 hours beforehand, as vendor ILG failed even to get through a successful run-through of the exam without encountering a range of tech issues. A recent law school grad conducted an experiment apparently showing that the facial recognition software struggled to differentiate between a human face and a glass of iced coffee.
“Are you kidding me? Canceling 3 days before exam, postponed until some date in October???” tweeted Seattle University School of Law professor Robert Chang, founder of the Fred T. Korematsu Center for Law and Equality, soon after the late Sunday night announcement from the Florida Board of Bar Examiners. “This is cruel and unfair to test-takers. It is harmful to the public.”
Recent University of Miami graduate Johnny Carver tweeted in response to Chang that he’d already received “frantic calls/texts” from some examinees who lost their jobs because of the delay. Others, he predicted, will lose their health insurance or could even be evicted “due to debt directly related to the exam.”
The use of proctors to gain access to test takers’ computers, and the question of what might happen to the data that’s collected, also has raised concerns among some privacy advocates.
Some states have devised old-school answers to their testing woes. Nevada, Indiana, and Louisiana decided to hold open book tests, which were emailed to and from test takers, ending the need to rely on online proctoring and often-faulty software.
Michigan completed its online test July 28, but only after a delay the tech vendor, ExamSoft, claimed was spurred by a cyberattack. Some test takers doubted the explanation, suggesting the software may have been overwhelmed.
Evan Miller, who is preparing to take the online exam in California, said more than twenty times as many people will be using ExamSoft’s software on Oct. 5-6, than as took the test in Michigan.
“And they claim it’s going to work? I’m very skeptical.”
ExamSoft representative Nici Sandberg said in a written statement that its software can handle the volume. The company successfully delivers “hundreds of thousands of secure exams for thousands of client programs every month,” including more than 11 million exams in 2019, she said.
Sandberg referred questions about remote proctoring, and the impact that exam policies may have on disabled and pregnant test takers, back to the states.
Representatives from ILG, including executive vice president Joseph Figo, did not respond to emailed questions.
‘Eyes Wide Open’
States are demanding that test takers obey of number of rules to make sure they don’t get flagged by artificial intelligence software as engaging in possible use of electronic devices while taking the test, or other forms of cheating. That includes not leaving the view of the webcam, and “not having the webcam centered on your face completely,” according to the rules set out by the State Bar of California, for example.
State administrators said AI software won’t decide who may be cheating.
ExamSoft, California’s bar exam vendor, “assures us that the software facial recognition works for all test-takers, but humans will review the videos after the exam is over,” said Merrill Balassone, a spokeswoman for the Judicial Council of California.
No AI software will be used to reject or terminate the exam session at any time during the exam, Balassone said. “Instead, the software will generate flags that will later reviewed by humans.”
Several large states, including New York and Pennsylvania, haven’t yet said which vendor they’ll be using for the software.
David Fine, chair of the Pennsylvania Board of Law Examiners, said he expects whichever vendor the state chooses will utilize remote proctoring and facial recognition software.
“Board staff have administered in-person exams for years and have experience watching typical test behaviors,” said Fine, a K&L Gates partner based in Harrisburg, Pennsylvania. “They will apply this background to the review of any applicant’s conduct that a proctor identifies.”
Fine added that the board anticipates that the software it uses “will not require a constant internet connection,” which should reduce the likelihood of malfunctions.
Judy Gundersen is president and CEO of the National Conference of Bar Examiners, which devises exam content and works with states on test administration. She believes the states and vendors will be able to work out any potential glitches before early October.
“I have confidence in the administrators who are working night and day” to ensure successful exams, she said. “Everyone is going into this with their eyes wide open.”