Foundations for Practice: Hiring the Whole Lawyer: Experience Matters

In July 2016, we published Foundations for Practice: The Whole Lawyer and the Character Quotient,(1) which shared findings from a survey that asked more than 24,000 lawyers what new lawyers need as they enter the profession. Respondents overwhelmingly indicated that new lawyers need characteristics, alongside professional competencies and legal skills (collectively, “foundations”). New lawyers, it turns out, are successful when they can demonstrate much more than their intelligence and legal competency. We called the new lawyer who can demonstrate this combination of characteristics, competencies, and skills the “whole lawyer.”

Hiring the Whole Lawyer

If we want new lawyers to develop the requisite foundations, we of course need law schools to commit to admitting, educating, and graduating students who can demonstrate they have those foundations. We also need employers to commit to hiring new lawyers based on their demonstration of those foundations—rather than mainly academic achievement.

How can employers hire new lawyers who have the desired foundations? This report provides answers.

In our survey, we asked respondents to identify the foundations new lawyers need to be successful in the respondent’s specific type of organization, specialty, or department. Then we asked them to consider the helpfulness of a set of hiring criteria in determining whether a candidate for employment has the foundations they identified as important. Notably, we did not ask them how they currently hire. We effectively asked how they would hire if they wanted to identify candidates with the necessary foundations.

Experience Matters

We learned that experience matters. While many employers in practice still rely on criteria like class rank, law school, and law review, our respondents indicated that if they wanted to hire people with the broad array of foundations they identified as important, they would rely on criteria rooted in experience, including legal employment, recommendations from practitioners or judges, legal externships, participation in a law school clinic, and other experiential education.

In The Whole Lawyer and the Character Quotient, we recommended that law schools and the profession work together to ensure that new lawyers have the foundations they need to practice. Our findings here give them a place to start. While we do not believe there is only one way to ensure new lawyers have the foundations they need to be whole lawyers, we do believe the path toward a system that prepares lawyers who are ready to enter the profession will be elevated and supported by experience-focused learning and hiring.

1.  ALLI GERKMAN & LOGAN CORNETT, FOUNDATIONS FOR PRACTICE: THE WHOLE LAWYER AND THE CHARACTER QUOTIENT (2016), available at http://iaals.du.edu/sites/default/files/documents/publications/foundations_for_practice_whole_lawyer_character_quotient.pdf [hereinafter THE WHOLE LAWYER].

The challenges new lawyers face when seeking employment today are well documented. In 2015, just 60% of graduating students found employment as lawyers, with another 11% finding law-related positions. Nearly 25% of graduates did not find any employment—including professional and non-professional jobs.(2) Students seem to have better employment outcomes when they can demonstrate achievement on traditional metrics: perceived prestige of law school attended, class rank, and law review experience. Evidence of this is especially pronounced when we examine employment outcomes for students based on the law school attended. The top ten schools for employment rates(3) also occupy spots near the top of the rankings published by U.S. News & World Report,(4) which dominates law school rankings.

Top Ten Schools by Employment Rates (U.S. News & World Report Ranking)

Meanwhile, employers, including those hiring top students with law review experience from top schools, are dissatisfied with the preparation of new lawyers. In one survey, 95% of hiring partners and associates said they believe new lawyers lack key practical skills at the time of hiring.(5) In another, only 23% of practitioners said they believe new lawyers have sufficient skills to practice.(6) Bar organizations across the country have publicly grappled with the skills gap, including the American Bar Association.(7) Meanwhile, the media have reported countless stories with the same basic punchline: “What they taught us at this law firm is how to be a lawyer. . . . What they taught us at law school is how to graduate from law school.”(8)

This perceived skills gap may suggest that law schools are, in fact, falling short when preparing their students for practice. It may also suggest that legal employers are falling short when it comes to developing hiring practices that result in good hires. It is likely a combination of these problems, both of which the Foundations for Practice project seeks to address.

Endnotes:

2.  These numbers reflect long-term/full-time employment outcomes for 2015 graduates 10 months after graduation. The American Bar Association defines a “professional position” as “one that requires professional skills or training but for which a JD is neither required nor a demonstrable advantage,” and it defines a “non-professional” position as one that “does not require any special professional skills or training.” AM. BAR ASS'N SECTION OF LEGAL ED. – EMPLOYMENT SUMMARY REPORT, http://employmentsummary.abaquestionnaire.org/ (for numbers, select 2015 class under “Compilation-All Schools Data”; for definitions, select “2015 Questionnaire Definitions &Instructions”).

3.  These rankings are based on positions that are full-time, long-term, and require bar passage.

4.  U.S. NEWS AND WORLD REPORT, BEST GRAD SCHOOLS 2016 EDITION 100 (2015).

5.  LEXIS NEXIS, HIRING PARTNERS REVEAL NEW ATTORNEY READINESS FOR REAL WORLD PRACTICE 1 (2015), available at https://www.lexisnexis.com/documents/pdf/20150325064926_large.pdf.

6.  THE BARBRI GROUP, STATE OF THE LEGAL FIELD SURVEY 6 (2015), available at http://www.thebarbrigroup.com/files/whitepapers/220173_bar_research-summary_1502_v09.pdf.

7.  AM. BAR ASS'N TASK FORCE ON THE FUTURE OF LEGAL ED., REPORT AND RECOMMENDATIONS (2014), available at http://www.americanbar.org/content/dam/aba/administrative/professional_responsibility/report_and_recommendations_of_aba_task_force.authcheckdam.pdf.

8.  David Segal, What They Don’t Teach Law Students: Lawyering, N.Y. TIMES, Nov. 20, 2011, at A1, available at http://www.nytimes.com/2011/11/20/business/after-law-school-associates-learn-to-be-lawyers.html.

In late 2014, we launched Foundations for Practice, a national, multi-year project designed to:

1. Identify the foundations entry-level lawyers need to launch successful careers in the legal profession;
2. Develop measurable models of legal education that support those foundations; and
3. Align market needs with hiring practices to incentivize positive improvements in legal education.

To meet the first objective, we developed a national survey to ascertain the legal profession’s perspective on the legal skills, professional competencies, and characteristics (collectively, “foundations”) that new lawyers need to succeed. Then, in partnership with state bar organizations across the country and generous individuals willing to champion the effort, we administered the survey in 37 states during the fourth quarter of 2014 and the first quarter of 2015. The survey was sent to an estimated 780,694 lawyers, and a total of 24,137 lawyers—with office locations in all 50 states and representing most practice settings and areas of expertise—submitted valid responses.(9)

The survey included two substantive sections. First, we presented a list of 147 foundations(10) and asked respondents to indicate, for their specific type of organization, specialty, or department, whether the foundation was:

• “Necessary immediately for the new lawyer’s success in the short term” (where “new lawyer” was defined as “someone embarking on their first year of law-related work”);
• “Not necessary in the short term but must be acquired for the lawyer’s continued success over time”;
• “Not necessary at any point but advantageous to the lawyer’s success”; or
• “Not relevant to success.”

The results of this section of the survey were documented in a report we released in July 2016.(11) In short, respondents definitively said that new lawyers need character. “When we talk about what makes people—not just lawyers—successful we have come to accept that they require some threshold intelligence quotient (IQ) and, in more recent years, that they also require a favorable emotional intelligence (EQ). Our findings suggest that lawyers also require some level of character quotient (CQ).”(12) More broadly, respondents indicated that new graduates must have a combination of characteristics, professional competencies, and legal skills comprising the “whole lawyer.”(13)

The survey was designed to give us a broader view into exactly what new lawyers need as they embark on their careers, but from the beginning we knew that in order to make use of this information, we needed a better understanding of how hiring happens—or, rather, how hiring would look if the profession hired based on the foundations it desired. This informed our development of the second section of the survey, which is the focus of this report. In that section, we asked respondents to indicate the helpfulness of seventeen distinct hiring criteria in determining whether a candidate for employment has the foundations they identified as important; that is, rather than asking about current hiring practices or what experiences or accomplishments they value in hiring, we effectively asked them to reflect on their responses to the 147 foundations presented in the survey and to indicate which of the seventeen hiring criteria presented would be useful in determining whether a candidate possesses the necessary foundations:(14)

• Law school attended

• Class rank

• Law review experience

• Journal experience

• Legal employment

• Legal externship

• Participation in law school clinic

• Other experiential education

• Law school courses in a particular specialty

• Law school certification in a particular specialty

• Recommendations from professors

• Recommendations from practitioners or judges

• Extracurricular activities

• Life experience between college and law school

• State court clerkship

• Federal court clerkship

• Ties to a particular geographic location

All Criteria Are Helpful

Unsurprisingly, respondents indicated that the full list of criteria was helpful in hiring. In fact, only one of the seventeen criteria was considered helpful by less than half of respondents. Further, the largest proportion of respondents to indicate any one criterion is either somewhat or very unhelpful was 14%.

Still, respondents conveyed preferences for some criteria over others. Differences in the helpfulness of criteriae merged, particularly with respect to the proportion of respondents who categorized the criteria as very helpful contrasted with the proportion who categorized the criteria as neither helpful nor unhelpful. Indeed, the proportions of respondents who selected very helpful ranged from 9% to 54%; similarly, the proportions of respondents who selected neither helpful nor unhelpful ranged from 6% to 38%. The proportions of respondents selecting other options remained relatively stable for all seventeen criteria.

Figure 1: Helpfulness of All Hiring Criteria

Trending Toward Practical Experience

To help conceptualize the results, we have grouped the criteria into three categories. The table below presents the hiring criteria as we categorized them.

Table 1: Hiring Criteria Categories

While all criteria were found to be far more helpful than unhelpful, one trend was notable. Respondents tended to rate criteria related to practical experience as more helpful than other criteria. Indeed, eight of the top ten most helpful criteria fall under the practical experience umbrella. Only two of the top ten most helpful hiring criteria were related to academic experience or achievement. This suggests that hiring lawyers tend to view experience actually working in the law as indicative of a new lawyer possessing the foundations necessary for success. This raised the question: do the seresults change when we look at responses across certain respondent demographic characteristics?

Figure 2: Top Ten Most Helpful Hiring Criteria

Endnotes:

9.  For a full report on our survey methodology, see ALLI GERKMAN & LOGAN CORNETT, FOUNDATIONS FOR PRACTICE: SURVEY OVERVIEW AND METHODOLOGICAL APPROACH (2016), available at http://iaals.du.edu/sites/default/files/documents/publications/foundations_for_practice_survey_overview_and_methodological_approach.pdf.

10.  “Foundations” include legal skills, characteristics, and professional competencies.

11.  THE WHOLE LAWYER, supra note 1, at 4.

12.  Id. at 1.

13.  Id.

14.  The survey also included a “Don’t Know” response option for the hiring criteria items. Respondents who selected “Don’t Know” for a given criterion have been removed from analysis for that criterion.

The following sections explore variations in the helpfulness of hiring criteria based on certain respondent demographic characteristics.(15) Specifically, we examine:(16)

• Practice setting;
• Firm size (for those in private practice);
• Years of experience; and
• Geographic region.

Notably, across all of these groups, responses were much more similar than different—respondents tended to view each criterion as helpful, regardless of their specific demographic characteristics. Still, analysis yielded a large number of statistically significant results. A statistically significant result tells us that there is a relationship between the variables—a specific hiring criterion and the analyzed demographic group—which likely reflects a true response pattern in the population, rather than emerging from the sample by chance. However, when respondents in one demographic group consistently indicate that a given hiring criterion is more (or less) helpful than do those in another group and when there are a large number of responses, a result may be statistically significant even when there is not much divergence between the groups. For example, a difference of one or two percentage points between two groups may be statistically significant, but such a small difference is not large enough to indicate a meaningful difference upon which law schools and law students might base a change in curricula or focus.

Accordingly, and in addition to statistical significance alone, we have determined that we will consider a result practically significant if the difference between demographic groups is large enough to influence our thinking in practice. Specifically, for the purposes of this report, we have defined a practically significant difference as one ten percentage points or larger between any two demographic groups.(17)

We also wanted to explore and compare the ten hiring criteria within each demographic group which respondents identified as the most helpful in determining whether a new lawyer possesses the needed foundations. Together with the discussion of practical significance, we are able to see the full picture of how different demographic groups view the helpfulness of each criterion differently—or similarly.

Endnotes:

15.  We used chi-square analyses to test the relationships between demographic characteristics and helpfulness of hiring criteria. To adjust for the large number of statistical tests conducted, we used a statistical significance level of p < .003.

16.  One potential concern in analysis was whether respondents with a role in hiring new lawyers would respond differently from those who do not currently have such a role. Thus, in addition to the four demographic characteristics presented in this report, we analyzed the hiring criteria data by respondent role in hiring. The results of this analysis showed no practically significant relationship between hiring role and any of the seventeen hiring criteria. We can be, therefore, confident that responses are consistent between those who hire new lawyers and those who do not.

17.  All results are presented in the appendix, available at http://iaals.du.edu/sites/default/files/documents/publications/hiring_whole_lawyer_appendix.pdf.

We analyzed survey responses to identify any differences in the helpfulness of hiring criteria based on a respondent’s practice setting. The survey allowed respondents to identify their practice setting from a list of 23 options.(18) However, in order to produce more meaningful, digestible results, we collapsed these options into four groups: private practice, business in-house counsel, government, and other settings (e.g., education, legal services/public defender, public interest/nonprofit).(19) A majority of respondents worked in private practice (58%), while much smaller proportions worked in government (18%), other (17%), and business in-house (8%) settings.

Figure 3: Respondent Practice Setting (n = 23080)

For respondents in private practice, government, and other settings, the top ten most helpful hiring criteria included all eight practical experience criteria. In addition, the top ten for each of these three practice settings included two academic experience or achievement criteria—for private practice and government settings, these academic criteria took the ninth and tenth places, while they took sixth and tenth place for other settings. For business in-house respondents, however, three academic criteria made the top ten and took the fifth, ninth, and tenth places; the remaining seven criteria were in the practical experience category.

Figure 4: Top Ten Hiring Criteria for Practice Setting

More than 60% of respondents across practice settings considered each of the eight practical experience criteria to be helpful in hiring new lawyers. Those in private practice and government were considerably more likely to find federal and state court clerkships helpful, compared to their counterparts in business in-house and other settings. A similar pattern emerged for recommendations from judges or practitioners. Respondents in private practice were more likely than those in all other settings to report ties to a geographic location are helpful. Those in business in-house settings were somewhat less likely than those in other groups to find recommendations from professors to be a helpful criterion in determining whether a new lawyer has the needed foundations.

Of the seventeen hiring criteria, we found that ten had a statistically, but not practically, significant relationship with practice setting, while the remaining seven were both statistically and practically significant—no results were found to be neither practically nor statistically significant.

Table 2: Statistical and Practical Significance in Practice Setting Analysis

Figure 5: Proportion of Respondents Indicating Somewhat or Very Helpful for All Hiring Criteria, Practice Setting

Endnotes:

18.  Respondents who indicated their practice setting was either private practice, business in-house, or non-profit in-house were further asked to select which of nine banded response options reflected the number of lawyers in the firm or department.

19.  See the appendix for a full list of response options and how they fit into each of the four analysis groups, available at http://iaals.du.edu/sites/default/files/documents/publications/hiring_whole_lawyer_appendix.pdf.

In addition to examining the broader differences among all practice settings, we were interested in examining any differences that might exist among private practice firms of varying size. Of those respondents who reported working in a private practice setting, about one-third (33%) were in solo practice, while a slightly larger proportion worked in small firms (39%); smaller proportions worked in medium (18%) or large (10%) firms.

Figure 6: Private Practice Firm Size (n = 13313)

For all firm sizes, the top ten most helpful hiring criteria included all eight practical experience criteria, as well as two academic experience or achievement criteria. However, there was substantial variation in the placement of these criteria within each top ten list—especially with respect to the academic criteria, which were notably higher on the large firm list than the others.

Figure 7: Top Ten Hiring Criteria for Private Practice Firm Size

Across firm sizes, at least 70% of respondents found each of the practical experience criteria to be helpful in hiring new lawyers. With respect to the traditional hiring criteria—class rank, law school attended, and law review experience—the proportion of respondents indicating the criterion is helpful increases as firm size increases. A similar pattern emerged for journal experience, federal court clerkships, and ties to a particular geographic location. Conversely, in terms of participation in a law school clinic, law school courses in a particular specialty, and law school certification in a particular area, respondents were less likely to find these criteria helpful as firm size increased.

Analysis by private practice firm size demonstrated both statistically and practically significant results for eleven of the seventeen hiring criteria. Five results were statistically, but not practically significant, while one was neither.

Table 3: Statistical and Practical Significance in Private Practice Firm Size Analysis

Figure 8: Proportion of Respondents Indicating Somewhat or Very Helpful for All Hiring Criteria, Private Practice Firm Size

We analyzed survey responses to identify any differences in the helpfulness of hiring criteria based on a respondent’s years of experience as a lawyer.(20) Respondents were somewhat more concentrated on the lower and higher ends of the experience spectrum, with 29% having 1-10 years of experience and 31% having more than 31 years; about one in five respondents had either 11-20 years (20%) or 21-30 years (21%) experience.

Figure 9: Years of Experience (n = 23099)

Analysis by respondent years of experience largely demonstrates consistency in respondents’ views of the helpfulness of each of the hiring criteria. The top ten hiring criteria across all years of experience categories consisted of all eight practical experience criteria, with two academic experience or achievement criteria taking the ninth and tenth places.

Figure 10: Top Ten Hiring Criteria for Years of Experience

Notwithstanding the similarities in the years of experience top ten lists, generally speaking, respondents with more than 31 years of experience are somewhat less likely than other groups to consider any of the hiring criteria helpful, although each criterion was considered helpful by a majority of these more experienced respondents. Notably, however, lawyers with more than 31 years of experience were more likely than those with less experience to view the more traditional hiring criteria—class rank, law school attended, and law review experience—as helpful.

Of the seventeen hiring criteria, we found that two yielded no significant results; eleven yielded statistically, but not practically significant, results; and four yielded both statistically and practically significant results when analyzed by years of experience.

Table 4: Statistical and Practical Significance in Years of Experience Analysis

Figure 11: Proportion of Respondents Indicating Somewhat or Very Helpful for All Hiring Criteria, Years of Experience

Endnotes:

20.  The survey asked respondents to provide their year of graduation from law school. During analysis we calculated the years between each respondent’s graduation year and 2015; we used this value as a proxy for years of experience. Further, we collapsed years of experience into four banded categories: 1-10 years; 11-20 years; 21-30 years; and 31+ years.

Finally, we were interested in identifying any differences in the helpfulness of hiring criteria based on a respondent’s geographic region.(21) About one-third of respondents reported practicing in the Midwest (32%) and another third practiced in the South (32%); about one-quarter practiced in the West (24%), while the remainder (12%) practiced in the Northeast.(22)

Figure 12: Region (n = 24072)

Analysis by respondent region largely demonstrates consistency in respondents’ views of the helpfulness of each of the hiring criteria. The top ten most helpful hiring criteria in each region include all eight practical experience criteria, with two academic experience or achievement criteria occupying the last two slots.

Figure 13: Top Ten Hiring Criteria for Region

In further demonstration of consistency across regions, more than 70% of respondents in each region considered each of the practical experience criteria to be helpful. Moreover, there is little variation in the proportion of respondents in each region indicating each criterion is either somewhat or very helpful.

Analysis by geographic region showed that, of the seventeen hiring criteria, there were no significant results for four criteria; there were statistically, but not practically, significant results for the remaining thirteen criteria. There were no results that were both statistically and practically significant.

Table 5: Statistical and Practical Significance in Region Analysis

Figure 14: Proportion of Respondents Indicating Somewhat or Very Helpful for All Hiring Criteria, Region

Endnotes:

21.  Geographic regions are based upon US Census regions. Regions and Divisions, U.S. CENSUS BUREAU, http://www.census.gov/econ/census/help/geography/regions_and_divisions.html.

22.  Respondents who reported practicing outside the United States (0.3%) were removed from this analysis.

The current discourse about the preparation of lawyers places much of the problem on the shoulders of law schools, which are commonly criticized as focusing more on teaching students how to think like lawyers than how to actually be lawyers. Indeed, law schools have a duty to equip students with the characteristics, professional competencies, and legal skills they will need as new lawyers, but they are not the only stakeholders that need to act. Legal employers who want new lawyers to have specific characteristics, professional competencies, and legal skills also have a duty to implement hiring practices that favor job candidates who possess those foundations.

By considering the findings in this report, legal employers can begin to do that, and law schools, law students, and recent graduates can gain insight into the types of experiences and accomplishments that may signal to employers that they have the whole package—or, that they are “whole lawyers.”

Experience Matters

The results are clear. Experience matters. Overall, respondents identified the eight hiring criteria related to practical experience(23) as the most helpful in determining whether a new lawyer possesses the foundations necessary for success. Moreover, while we see these criteria shift across the different respondent groups we examined, in eleven of the sixteen individual analysis groups they still occupy the top eight slots (even if ordered differently), including two practice settings (private practice and government), one firm size (2-10 lawyers), and all years of experience and region groups. In four of the five remaining analysis groups, the eight practical experience hiring criteria still occupy positions in the top ten criteria. The only analysis group that deviates from this pattern is business in-house counsel (under practice setting), and seven of the eight practical experience hiring criteria are still included in their top ten list.

While the hiring criteria shift positions on the top ten lists depending on analysis group, there is one notable exception: legal employment. In addition to being identified as the most helpful criterion in the overall analysis, respondents in every single demographic group we examined identified legal employment as the most helpful of all the hiring criteria. Across practice settings and firm sizes, years of experience and practice region, the respondents told us: there is no substitute for actual experience in a legal setting.

Endnotes:

23.  In this order: legal employment, recommendations from practitioners or judges, legal externships, other experiential education, life experience between college and law school, participation in law school clinic, federal court clerkship, and state court clerkship.

In our first report(24) on the survey results, we called on law schools and the profession to work together to ensure that new graduates have the necessary foundations to enter the profession. We recommended that law schools work with employers and the legal community to develop measurable learning outcomes—and create and reward law school programs and courses that develop the requisite personal characteristics, professional competencies, and legal skills. We also recommended that legal employers prioritize the characteristics, competencies, and legal skills they value in their hiring practices.

The emphasis on experience that we described in this report is both heartening and actionable. It suggests that as schools and the profession begin to consider 1) how to ensure law graduates have the necessary foundations and 2) how to hire new lawyers based on those foundations, they can start with experience. Law schools and employers may develop a common language for those experiences that clarifies what they are and how they benefit students. Experience may happen in a typical legal setting, in a clinic, or in a classroom. It may happen as part of the standard curriculum or as a supplement to the curriculum. It may become the primary criterion in hiring or it may supplement an employer’s existing criteria. While we do not believe there is only one way to ensure new lawyers have the foundations they need to be whole lawyers, we do believe the path toward a system that prepares lawyers who are ready to enter the profession will be elevated and supported by experience-focused learning and hiring.

Endnotes:

24.  THE WHOLE LAWYER, supra note 1.