University of Denver

Bar Examiners Task Force Releases Phase 2 Report on National Practice Analysis

Director of Research

In 2018, the National Conference of Bar Examiners (NCBE) created its Testing Task Force to conduct a three-year project that, in part, examines whether the bar exam is a valid measure of minimum competence to practice law. Phase 1 involved a series of listening sessions with stakeholders to gather their impressions about the current bar exam, as well as ideas for what the exam could look like moving forward. In August 2019, NCBE released its Phase 1 report, which one observer called “a forceful critique” that “showcases an array of dissatisfied stakeholders who directly question whether the exam actually measures minimum competence at all.” Simultaneous to that report, NCBE kicked off Phase 2—a national practice analysis—to provide a foundation of data for bar exam reform, which NCBE will consider in Phase 3 of the project. In late March 2020, NCBE released a report detailing its findings.

The national practice analysis aimed to provide empirical data on the job activities of newly licensed lawyers (NLLs), defined as being licensed for three years or less. The survey garnered nearly 15,000 valid responses, 21 percent of whom were NLLs; the balance were non-NLLs with experience directly supervising NLLs. Respondents came from 56 jurisdictions and reflected a variety of experience levels, practice areas, and work settings.

The survey consisted of five sections:

  • Demographics and practice areas
  • Tasks
  • Knowledge areas
  • Skills, abilities, and other characteristics (SAOs)
  • Technology

NCBE divided the survey into four separate versions, each of which included the demographic questions and a few subsections of the broader survey. Respondents were randomly assigned to one of the four versions.

Tasks

Most Common Least Common
  • Identify issues in client matter, including legal, factual, or evidentiary issues
  • Research case law
  • Interpret laws, rulings, and regulations for client
  • Research statutory and constitutional authority
  • Evaluate strengths and weaknesses of client matter
  • Establish and maintain client trust account
  • Participate in initaitve or proposition process to change statute of constitution
  • Represent client in post-conviction relief or habeas corpus proceedings
  • Represent client in eminent domain or condemnation proceeding
  • Draft constitutional amendments

Knowledge Areas

Highest Mean Importance Lowest Mean Importance
  • Rules of professional responsibility and ethical obligations
  • Civil procedure
  • Contract law
  • Rules of evidence
  • Legal research methodology
  • Transportation law
  • Bioethics
  • Public utility law
  • Sports and entertainment law
  • Admiralty law

SAOs

Highest Mean Criticality Lowest Mean Criticality
  • Written/reading comprehension
  • Critical/analytical thinking
  • Written expression
  • Identifying issues
  • Integrity/honesty
  • Strategic planning
  • Leadership
  • Social consciousness/community involvement
  • Networkign and business development
  • Instructing/mentoring

Technology

Highest Mean Proficiency Lowest Mean Proficiency
  • Word processing software
  • Research software/platforms
  • Electronic communication software
  • Desktop publishing software
  • Document storage software
  • Web content management software
  • Data analytics software
  • Language translation software
  • Financial planning software
  • Tax preparation software

Using empirical data to discover what new lawyers need for competent entry-level law practice is at the heart of IAALS’ Building a Better Bar project. A research-based definition of minimum competence will help align the bar exam with the consumer protection goals the licensing process seeks to achieve. We have, at NCBE’s invitation, shared preliminary findings from Building a Better Bar with them to facilitate understanding and interpretation of their practice analysis data. We are excited to continue our work, complementing research by NCBE and others, to ensure the legal profession is made up of competent lawyers who can serve their clients well.

Mentioned Content 
Project
The legal profession has never had a clear, explicit understanding of the minimum competence needed to practice law and how it should be tested on the bar exam (or through other licensing approaches). Through this project, we have defined minimum competence and have new recommendations for how the legal licensing process—...Read More