In keeping with a strategic goal to ensure that its admission system is timely and fair, the State Bar of California has conducted a comprehensive series of studies of the California Bar Exam. The work has been prompted by several factors, including a multiyear decline in pass rates. This trend raised the question of whether the current pass line, or cut score, of 1440 is appropriate.
The current pass line was established over three decades ago. Critics point out that California’s pass line is higher than every other state in the country except for Delaware. Others contend that factors other than the cut score influence pass rates.
The California Supreme Court, which has ultimate authority over the Bar Exam and cut score, directed the agency to ensure that these studies:
The first three studies were completed in 2017. Each study was led by an outside consultant with nationally recognized expertise in the subject. In addition, the State Bar hired additional subject matter experts to serve as external reviewers of the studies’ methods and findings. A working group comprised of a single representative from the California Supreme Court, and two representatives each from State Bar Board of Trustees and the Committee of Bar Examiners, oversaw the work.
The 2017 studies included:
After completing the standard-setting study, the State Bar invited public comment, surveyed practicing attorney and applicants, and conducted public hearings. In September 2017, the State Bar issued a report to the California Supreme Court, which has ultimate authority over the California Bar Exam cut score.
The State Bar Board of Trustees offered three options for the Supreme Court’s consideration:
The report also outlined key issues related to the Court's policy decision – public protection, access to justice and diversity.
In October 2017, the Supreme Court issued its response, maintaining the current cut score for the time being, as the State Bar continued its research.
The fourth study, completed in 2018, examined changes in the characteristics of students taking the California Bar Exam to provide a better understanding of the declining trend of the bar passage rates. The study found that changes over time in the characteristics of exam takers accounted for between 20 and 50 percent of the decline in bar exam performance during the study period. The study was unable to account for a substantial amount of the decline in pass rates, concluding that other unexamined factors have contributed to the decade-long decrease in bar exam performance.
Dr. Roger Bolus led the study. An advisory group of law school deans participated throughout the project. The study examined detailed data on over 7,000 students (from 11 ABA law schools participating in the study) who took the California Bar Exam in July 2013, 2016, and 2017.
Individual data included information on students’ undergraduate grade point average (GPA), undergraduate major, LSAT score, final law school GPA, course work in law school, and basic demographic information. The study examined these characteristics in relation to bar exam performance—both pass/fail outcomes and scores. The study found that:
Read more
In December 2018, the State Bar began the first California-specific study of the knowledge and skills needed for entry-level attorneys. The study collected detailed, empirical data about how attorneys use their knowledge and skills to perform routine tasks in the practice of law. A working group, with members selected by the California Supreme Court from state and national stakeholder groups, oversaw the study.
The working group’s final report contains three broad recommendations designed to bring the California Bar Exam into closer alignment with the current practice of law for entry-level attorneys in California:
Fact Sheet: California Attorney Practice Analysis
In addition to CAPA, the State Bar completed two other bar exam studies in 2019:
Differential Item Functioning Analysis, which studied the potential differential impact of exam questions by race, gender, and other factors. Examining 20 bar exams from July 2009 to February 2019, the study found no major areas of concern, but recommended further action to continue improving the exam by eliminating sources of differential impact.
Evaluation of Grading Processes—As an outgrowth of recommendations from the 2017 Governance in the Public Interest Task Force, the State Bar’s consulting psychometrician evaluated the State Bar’s multiphase exam grading process to identify potential efficiencies and best practices while continuing to ensure necessary consistency in the grading process.
These studies, along with a review of administrative procedures conducted by the California Department of Consumer Affairs, were summarized in a report to the Board of Trustees presented in May 2020.
In March 2020, the State Bar published a simulation of the impact of different bar exam cut scores on bar passage, by gender, race/ethnicity, and law school type. The simulation was based on results from 21 bar exams administered over 11 years, from February 2009 to February 2019. The simulation compared the number of exam takers among the various subgroups who would have passed the bar exam if the cut score had been 1300, 1330, 1350, and 1390. The smallest reduction in the cut score increased the overall pass rate by 8 percentage points, while the greatest reduction in cut score increased it by 31 percentage points, with wide-ranging impacts depending on the subgroup.
For questions contact: Ron Pi, Principal Program Analyst, Office of Research & Institutional Accountability, 415-538-2013, ron.pi@calbar.ca.gov