Bloomberg Law
July 16, 2020, 8:00 AM

INSIGHT: Saving the Bar Exam By Focusing on Performance

Jason Solomon
Jason Solomon
Stanford Law School

Some states are still scheduled to have in-person bar exams in July, but other state bars are grappling with the immediate and urgent question of whether to proceed, or what to do instead to license new lawyers.

In this pandemic emergency, there are no perfect options. No matter what happens this year, though, the question remains: what to do about the much-criticized bar exam? Indeed, the debate over what to do in an emergency is illustrative of what we think of the bar exam and its future. Whether we’re willing to throw the bar exam overboard as we approach the iceberg depends on our attachment before the storm hit.

It also depends on what we think of the alternatives, and the proposed alternatives for the crisis and beyond—diploma privilege and supervised practice—have significant flaws themselves.

As to diploma privilege, the National Conference of Bar Examiners (NCBE) has a point: law school curricula have not been designed to ensure minimum competence, though the ABA has recently pushed in this direction. And requiring students to take certain courses does not tell you much about competence.

As to supervised practice, it is certainly a logical approach to the crisis. But long-term, we would have to think carefully about returning to an apprenticeship system.

The U.S. used to require this for bar admission, but apprentices often got lots of mundane tasks and little mentoring. Today, older and younger lawyers alike often decry the lack of time for mentoring and teaching new lawyers. It is not clear that experienced lawyers will both make the time to assure that new lawyers are minimally competent, and be willing to tell the bar to turn them down if necessary.

Consensus on Immediate and Necessary Reforms

The good news is that there is significant consensus among experts on the bar exam around the most immediate and necessary reforms, should it remain in place in some fashion. It should be open-book, not closed-book. In an era where information is readily available at our fingertips, memorization of doctrine is simply not a skill that is important for today’s lawyers.

There should be fewer topics tested, a logical conclusion from both the NCBE and California Bar’s recent job-validation studies that looked at the doctrinal areas most used by lawyers. And to the extent that some test of doctrinal knowledge is retained, it should be after the first or second year of law school, closer to when that material is learned and based on the well-established model of medical licensing exams.

In the language of test designers, the core problem of the bar exam is its validity. Because the principal competency it tests on the multiple-choice and essay sections—applying doctrinal concepts from memory—is so unrelated to modern law practice, there is no reason to believe that it is an effective screen for minimum competence.

More Focus on ‘Performance’ Part of the Exam

The NCBE and California have recently completed important job-validation studies, but neither focuses on this competency of applying doctrine from memory. The good news is that the competencies that do emerge from the studies are tested in a less familiar part of the exam, the Multistate Performance Test (MPT)—added more than two decades ago, worth 20% of the Uniform Bar Exam, and part of the exam in most states.

The MPT consists of two simulated problems. Students are given a scenario, the relevant legal materials (cases, statutes), and documents, deposition excerpts, and other results from a factual investigation. And they are asked to come up with an analysis of how to address the situation, acting as a lawyer. The performance test is designed to assess six fundamental skills that stand out in studies of lawyer competencies: problem-solving, legal analysis and reasoning, factual analysis, communication, organization and management of a legal task, and recognizing and resolving ethical dilemmas.

Some performance tests are more typical of litigation practice or law school: “You’re a law clerk for a judge, write a memo on how you would decide this case.” But others are more challenging and reflect a range of lawyering tasks.

One test asked students to look at a draft liability waiver for a client opening a business, and suggest edits that would both advance the business’ interests and ensure the waiver’s enforceability. Another test asked examinees to evaluate the pros and cons of different estate-planning approaches and make a recommendation to the client in light of his goals.

Increased emphasis on the performance test should get support from both defenders and critics of the bar exam. Stakeholders in NCBE’s recent study pointed to the MPT as the strongest part of the bar exam. Rather than multiple-choice on memorized doctrine being the most heavily weighted portion, and performing actual lawyering tasks the least, the NCBE and state bars ought to try flipping the equation.

To be sure, the performance tests can and should improve. As Ben Bratman has pointed out, not all of the lawyering skills are assessed as well as others. More tests should ask examinees to advise clients at the planning or counseling stage, as opposed to dealing with disputes. And part of the performance test may need to be multiple choice to ensure reliability in grading.

Focusing on the performance test could also be an important lever for legal education reform. Greater emphasis on the MPT should mean a shift in pedagogy towards more simulation or business-school “case method” teaching where students are put in the role of the lawyer.

The performance test could also help drive further attention to regulatory work. Bar examiners could include more statutes and regulations on performance tests—as Pennsylvania does—and this ought to spur more schools to make statutory analysis and administrative law part of the core curriculum.

The difficult decisions around this crisis should not obscure the considerable consensus around how to improve the bar exam. And bar examiners should feel confident that an under-appreciated part of the existing exam—the performance test—can be a large part of the solution.

This column does not necessarily reflect the opinion of The Bureau of National Affairs, Inc. or its owners.

Author Information

Jason Solomon is the executive director of the Stanford Center on the Legal Profession and a Lecturer in Law at Stanford Law School. He previously served as the chief legal officer of the education nonprofit Summit Learning, and as a law professor at William and Mary and the University of Georgia.