In keeping with a strategic goal to ensure that its admission system is timely and fair, the State Bar of California conducted a comprehensive series of studies of the California Bar Exam. The work was prompted by several factors, including a multiyear decline in pass rates. This trend led the State Bar to evaluate the appropriateness of California's cut score, one of the highest in the nation.
The California Supreme Court, which has ultimate authority over the Bar Exam and cut score, directed the agency to ensure that these studies:
The first three studies were completed in 2017. Each study was led by an outside consultant with nationally recognized expertise in the subject. In addition, the State Bar hired additional subject matter experts to serve as external reviewers of the studies’ methods and findings. A working group comprised of a single representative from the California Supreme Court, and two representatives each from State Bar Board of Trustees and the Committee of Bar Examiners, oversaw the work.
The 2017 studies included:
After completing the standard-setting study, the State Bar invited public comment, surveyed practicing attorney and applicants, and conducted public hearings. In September 2017, the State Bar issued a report to the California Supreme Court, which has ultimate authority over the California Bar Exam cut score.
The State Bar Board of Trustees offered three options for the Supreme Court’s consideration:
The report also outlined key issues important for the Court's policy decision–public protection, access to justice, and diversity of the legal profession.
In October 2017, the Supreme Court issued its response, maintaining the current cut score for the time being, as the State Bar continued its research.
The fourth study, completed in 2018, examined changes in the characteristics of students taking the California Bar Exam to provide a better understanding of the declining trend of the bar passage rates. The study found that changes over time in the characteristics of exam takers accounted for between 20 and 50 percent of the decline in bar exam performance during the study period. The study was unable to account for a substantial amount of the decline in pass rates, concluding that other unexamined factors have contributed to the decade-long decrease in bar exam performance. Read More
Dr. Roger Bolus led the study. An advisory group of law school deans participated throughout the project. The study examined detailed data on over 7,000 students (from 11 ABA law schools participating in the study) who took the California Bar Exam in July 2013, 2016, and 2017.
Individual data included information on students’ undergraduate grade point average (GPA), undergraduate major, LSAT score, final law school GPA, course work in law school, and basic demographic information. The study examined these characteristics in relation to bar exam performance—both pass/fail outcomes and scores. The study found that:
Beginning in 2018, the State Bar has been working with a team of law and psychology researchers from Indiana University, University of Southern California, and Stanford University on an online program for bar exam applicants, intended to help them adopt a productive mindset while preparing for the exam. The Mindsets in Legal Education researchers designed, administered, and evaluated the intervention, known as the California Bar Exam Strategies and Stories Program. The online program includes an introductory film, stories from prior test takers, and a writing activity in which participants share insights and strategies that may be useful to them and to future test takers.
The program was first offered to all applicants for the July 2018 bar exam, and again to July 2019 bar exam applicants. Results from the first two years have been promising: the program increased the likelihood of participants passing the bar exam ranging between 6.8 to 9.6 percentage points, controlling for other factors. The impact was even higher for applicants in disadvantaged groups, including those in underrepresented racial/ethnic populations and those who are first-generation college students. The program was again offered to all bar exam applicants for the October 2020 bar exam.
In December 2018, the State Bar began the first California-specific study of the knowledge, skills, and abilities required by entry-level attorneys. The study collected detailed, empirical data about how attorneys use their knowledge and skills to perform routine tasks in the practice of law. A working group, with members selected by the California Supreme Court from state and national stakeholder groups, oversaw the study.
The working group’s final report contains three broad recommendations designed to bring the California Bar Exam into closer alignment with the current practice of law for entry-level attorneys in California:
Fact Sheet: California Attorney Practice Analysis
In addition to CAPA, the State Bar completed two other bar exam studies in 2019:
Differential Item Functioning Analysis, which studied the potential differential impact of exam questions by race, gender, and other factors. Examining 20 bar exams from July 2009 to February 2019, the study found no major areas of concern, but recommended further action to continue improving the exam by eliminating minor sources of differential impact.
Evaluation of Grading Processes—The State Bar evaluated its multiphase exam grading process to identify potential efficiencies and best practices in grading while continuing to ensure necessary consistency in the grading process.
These studies, along with a review of test administration procedures conducted by the California Department of Consumer Affairs, were summarized in a report to the Board of Trustees presented in May 2020.
In March 2020, the State Bar published a simulation of the impact of different bar exam cut scores on bar passage, by gender, race/ethnicity, and law school type. The simulation was based on results from 21 bar exams administered over 11 years, from February 2009 to February 2019. The study simulated what the pass rates for the bar exam would have been for various populations by race/ethnicity and gender. If the cut score had been 1300, 1330, 1350, or 1390 at the time of those exams. The smallest reduction in the cut score increased the overall pass rate by 8 percentage points, while the greatest reduction in cut score increased it by 31 percentage points, with wide-ranging impacts depending on the subgroup.
For questions contact: Ron Pi, Principal Program Analyst, Office of Research & Institutional Accountability, 415-538-2013, firstname.lastname@example.org