Photo credit: arstechnica.com
California Bar Exam Controversy: Use of AI Raises Concerns
On Monday, the State Bar of California announced its deployment of artificial intelligence in crafting some of the multiple-choice questions for the February 2025 bar exam. This news has sparked significant backlash from law school educators and participants, particularly in light of prior complaints regarding various technical issues and discrepancies during the exam’s administration, as noted by the Los Angeles Times.
The State Bar revealed that 23 out of the 171 scored questions were developed with AI support, through its psychometrician, ACS Ventures. An additional 48 questions originated from a first-year law exam, while the balance of 100 questions was designed by Kaplan Exam Services.
In defense of its strategy, the State Bar stated to the LA Times that all exam questions underwent rigorous scrutiny by panels tasked with content validation and by subject matter experts prior to the test. “The ACS questions were developed with the assistance of AI and subsequently reviewed by content validation panels and a subject matter expert in advance of the exam,” elaborated Leah Wilson, Executive Director of the State Bar, in a press statement.
This revelation has provoked considerable criticism from various legal education professionals. Mary Basick, assistant dean of academic skills at the University of California, Irvine School of Law, expressed her astonishment, stating, “The debacle that was the February 2025 bar exam is worse than we imagined. I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.”
Similarly, Katie Moran, an associate professor at the University of San Francisco School of Law specializing in bar exam preparation, described the situation as “a staggering admission.” She highlighted the potential conflict of interest, noting that the same entity responsible for developing the AI-generated questions was also tasked with their evaluation and subsequent approval for examination use.
California’s Legal Community Reacts to AI Question Development
In response to the growing criticism, Alex Chan, the chair of the State Bar’s Committee of Bar Examiners, emphasized that the California Supreme Court had encouraged the exploration of “new technologies, such as artificial intelligence” as a means to enhance the reliability and economic efficiency of testing processes.
Source
arstechnica.com