Bar Examiners Task Force Releases Phase 2 Report on National Practice Analysis
In 2018, the National Conference of Bar Examiners (NCBE) created its Testing Task Force to conduct a three-year project that, in part, examines whether the bar exam is a valid measure of minimum competence to practice law. Phase 1 involved a series of listening sessions with stakeholders to gather their impressions about the current bar exam, as well as ideas for what the exam could look like moving forward. In August 2019, NCBE released its Phase 1 report, which one observer called “a forceful critique” that “showcases an array of dissatisfied stakeholders who directly question whether the exam actually measures minimum competence at all.” Simultaneous to that report, NCBE kicked off Phase 2—a national practice analysis—to provide a foundation of data for bar exam reform, which NCBE will consider in Phase 3 of the project. In late March 2020, NCBE released a report detailing its findings.
The national practice analysis aimed to provide empirical data on the job activities of newly licensed lawyers (NLLs), defined as being licensed for three years or less. The survey garnered nearly 15,000 valid responses, 21 percent of whom were NLLs; the balance were non-NLLs with experience directly supervising NLLs. Respondents came from 56 jurisdictions and reflected a variety of experience levels, practice areas, and work settings.
The survey consisted of five sections:
- Demographics and practice areas
- Tasks
- Knowledge areas
- Skills, abilities, and other characteristics (SAOs)
- Technology
NCBE divided the survey into four separate versions, each of which included the demographic questions and a few subsections of the broader survey. Respondents were randomly assigned to one of the four versions.
| Most Common | Least Common |
|---|---|
|
|
| Highest Mean Importance | Lowest Mean Importance |
|---|---|
|
|
| Highest Mean Criticality | Lowest Mean Criticality |
|---|---|
|
|
| Highest Mean Proficiency | Lowest Mean Proficiency |
|---|---|
|
|
Using empirical data to discover what new lawyers need for competent entry-level law practice is at the heart of IAALS’ Building a Better Bar project. A research-based definition of minimum competence will help align the bar exam with the consumer protection goals the licensing process seeks to achieve. We have, at NCBE’s invitation, shared preliminary findings from Building a Better Bar with them to facilitate understanding and interpretation of their practice analysis data. We are excited to continue our work, complementing research by NCBE and others, to ensure the legal profession is made up of competent lawyers who can serve their clients well.
