The New York Board of Law Examiners is trying to allay concerns that the many strict rules surrounding the state’s online bar exam in early October are navigable, and won’t result in false allegations of cheating.
A frequently asked questions list released by the agency Tuesday also mandates that test takers sit for at least two mock exams before the actual test on Oct. 5-6, to make sure they understand the technology. Test takers also will need to learn how to use virtual “notes” instead of regular scrap paper, and meet strict deadlines to register their computers with the exam provider.
Concerns have been growing about the October tests being given in New York and 17 other states. In several other states, including Michigan, Florida, and Indiana, exam takers recently encountered significant problems with the software they used, or planned to use, through tech vendors ExamSoft or ILG Technologies. New York is using ExamSoft, which encountered issues in Michigan in late July, when test takers there were unable to log into a portion of the test because of an apparent cyberattack.
“This is not the test we prepared or signed up for,” advocacy group Diploma Privilege for New York tweeted in response to the FAQ. “This is not the test that thousands of lawyers took before us. This is a beta test for a format that isn’t going to work.”
“Diploma privilege” allows law school graduates who meet certain conditions to begin practicing without first taking a bar exam. The idea has gained steam due to public health concerns around coronavirus that have stymied traditional in-person testing in many states this year.
Privilege proponents take several issues with the New York test as laid out in the FAQs. These include concerns that the final scores will be reported on a scale that reflects the difficulty of this exam in comparison to other Multistate Bar Examinations—a complex system that reform advocates say unfairly muddies what should be clear-cut passing and failing scores. To pass, test takers will need a scaled score of at least 266 out of 400.
Another concern across several states is that the AI programs designed to determine whether or not a test taker is cheating on the online test might flag an individual for something innocuous like looking the wrong way, or having a child or pet enter the computer frame.
New York’s law examiner board says it’s taking those concerns seriously.
After each of the four 90-minute testing sessions, including two each day, video files will be uploaded to ExamSoft, so that its AI program called “ExamMonitoring” can analyze the recordings and flag “any unusual behaviors, movements, or sounds,” according to the FAQ.
Footage then will be reviewed by at least one human proctor, said the FAQ, in part so that everyday sounds like barking dogs or street sirens, which might be initially flagged by AI, can be disregarded as unsuspicious on review. BOLE says it anticipates most flags will be cleared by human proctors.
“The mere fact that a sound or behavior generates a flag does not mean that BOLE staff will formally investigate and/or take any action against an applicant for exam misconduct,” the FAQ said.
Mock Exam Questions
Mike Machado is a 2020 graduate of the University of Pennsylvania Law School. In an email to Bloomberg Law, he listed several reasons why he says New York’s bar exam plans, as detailed in the FAQ, are misguided.
ExamSoft’s software isn’t sufficiently tested, according to Machado. That’s especially problematic because many states, and their thousands of test takers, will be accessing the company’s tests simultaneously in early October, he said.
The current mock tests will not reveal flaws in ExamSoft’s software, said Machado, who is lined up to start work in Manhattan this fall as an associate with Morrison & Foerster.
“Based on the FAQ, these mocks will prove insufficient as a measure of ExamSoft’s capacity to administer the exam exactly how it plans to,” Machado said. “The only way to accurately test that is to simulate test-day conditions during the mock exams.”