Foundations for Practice: Survey Overview and Methodological Approach

Educating Tomorrow’s Lawyers (“ETL”) is an initiative of the Institute for the Advancement of the American Legal System (“IAALS”) dedicated to aligning legal education with the needs of an evolving profession. ETL fosters a constructive national dialogue among stakeholders, while conducting research to build informed approaches to improving legal education.

Foundations for Practice is a national, multi-year project of ETL designed to:

• Identify the foundations entry-level lawyers need to launch successful careers in the legal profession;
• Develop measurable models of legal education that support those foundations; and
• Align market needs with hiring practices to incentivize positive improvements.

We developed a national survey to ascertain the legal profession’s perspective on the skills, characteristics, and competencies that new lawyers need to succeed.

The stated goal of this first step in the multi-year Foundations for Practice research was to ascertain the legal skills, professional competencies, and characteristics—collectively referred to as foundations—that practicing attorneys identified as needed forentry-level lawyers to launch successful careers in the legal profession. While other researchers have conducted studies in a similar spirit, the Foundations for Practice survey is currently the most comprehensive effort undertaken, with respect to both its exhaustive content and its national scope.

Survey Development

Working from the existing literature in this area,(1) law firm core competencies obtained from employers, and input from a diverse group of experts in the field, we developed a survey capable of capturing the range of foundations new lawyers may need to be successful in their field and the degree to which those foundations are required, as well as an array of respondent characteristics and attributes. Thus, this instrument allows us to fully understand what the profession considers important, desirable, or inconsequential for new lawyers, and to dissect the information to determine how these foundations differ (or remain similar) across a variety of respondent groups.

Given the unique nature of this effort, one challenge we faced in developing the surveyinstrument was determining an appropriate set of response options for the individual foundations. A Likert-type scale placing agreement or importance on one end of the spectrum and disagreement or unimportance on the other—the most common type of scale employed in survey research—was not appropriate for this instrument, as such a scale would not allow us to adequately distinguish between foundations that must be developed by the time a student leaves law school and enters his or her career and foundations that may be developed after law school and over the course of the lawyer’s career. To overcome the apparent shortcomings of a more traditional survey scale, we created a novel ranking scale with response options including necessary in the short term, must be acquired over time, advantageous but not necessary, and not relevant.(2)

Another challenge in developing the survey instrument was balancing the desire to present a comprehensive list of foundations with the need to avoid duplication in the foundations included. Through a process of consulting the literature, seeking guidance from experts, and brainstorming, we began with more than 210 individual foundations in the original draft of the survey instrument. We were able to significantly decrease the number of foundations by ferreting out areas of overlap, then collapsing or removing items where appropriate. Importantly, rather than asking for general impressions of what foundations a person embarking upon their first year of law-related work needs, the survey instrument instructed attorneys to respond in the context of their specific type of organization, specialty, or department.

In addition to respondent attributes and assessments of the necessity of each foundation, we wanted to understand how attorneys viewed the helpfulness of the multitude of resources commonly used as hiring criteria for new attorneys, such as class rank, experiential education, clerkships, and recommendations. Finally, the survey instrument contained items by which attorneys could indicate their agreement with two statements related to the timing of specialization in a particular practice area, either during law school or in the first few years of practice.

Certainly, with the number of survey questions exceeding 170, the unique and monumental task undertaken in this survey effort required an instrument not for the faint of heart. Ultimately, the final version of the survey instrumentsought to capture the following from each attorney respondent:(3)

• Demographics and Practice-Specific Characteristics
  • Gender
  • Racial and ethnic background
  • Income
  • Primary work setting
  • Type of practice
  • Area of expertise
  • Interactions with new lawyers in the workplace
  • Office location
  • Years of experience
  • Law school attended
• Substantive Information
  • Degree of necessity for each of the 147 presented foundations
  • Helpfulness ranking for each of 17 hiring criteria
  • Appropriate timing of specialization in a particular practice area

Survey Distribution and Response Rate

In order to reach the largest possible number of attorneys, we worked with state bar organizations(4) in participating states to coordinate distribution of the survey to the entirety of their respective memberships. In total, bar associations in 37 states agreed to disseminate electronic links to the survey. Overall, bar organizations facilitated distribution of the survey instruments to an estimated 780,694 attorneys.(5)

Figure 1 presents participating states, as well as the number of attorneys who receivedan invitation to complete the survey in each state.(6)

Figure 1: Participating States and the Number of Survey Recipientsby State(7)

As alluded to above, the survey was administered using anonymous electronic survey links.(8) Generally speaking, a representative from within each participating organization sent an email to its constituency containing information about the survey and the link by which to complete it. However, in some cases, the state bar association included the link in a more broad-purpose communication, such as an e-newsletter. Distribution occurred on a rolling basis from November 13, 2014, to April 15, 2015, with Washington being the first state and Texas being the last state to distribute the survey link.

In total, we received 24,137 valid responses(9) to the survey, with respondents reporting office locations in all 50 states (as well as a number of territories and foreign locations).(10) Thus, the estimated response rate is 3.1%, with a margin of error of ±0.6% at a 95% confidence level. Note that, although the proportion of attorneys who responded to the survey would appear to be relatively low, the high number of responses yields a more-than-acceptable margin of error(11) at the conventional confidence level. This means that we can interpret the results with a high degree of confidence that the numbers reported are extremely close to what we would observe if we had responses from the full population.

Endnotes:

1.  We referred to Marjorie Maguire Shultz and Sheldon Zedeck, Final Report –Identification, Development and Validation of Predictors for Successful Lawyering (2009), available at http://ssrn.com/abstract=1353554 and ABA Section of Legal Education and Admissions to the Bar, Legal Education and Professional Development – An Educational Continuum (Report of the Task Force on Law Schools and the Profession: Narrowing the Gap) (1992), among others.

2.  The survey defined the response options as: necessary immediately for the new lawyer’s success in the short term; not necessary in the short term but must be acquired for the lawyer’s continued success over time; not necessary at any point but advantageous to the lawyer’s success; not relevant to success in this type of organization, specialty, or department.

3.  The Foundations for Practice survey is available at http://iaals.du.edu/sites/default/files/documents/publications/ffp_survey_paper_version.pdf.

4.  In California, seven local bar associations distributed the survey, as opposed to the State Bar of California.

5.  Unfortunately, the exact number of people who received an invitation to complete the survey cannot be known. First, as noted in a previous footnote, many states were unable to provide an exact count of the number of attorneys to whom they sent a survey link. Second, because thedistribution approach did not allow for sending unique survey links to each attorney, the intended recipients of the links may have further distributed it amongst their contacts; thus, potentially providing an avenue for attorneys who were not state bar association members to receive survey links.

6.  Some of the numbers presented in the map represent exact numbers, while others represent a best estimate of recipients provided by the bar organization for that state.

7.  The relatively low numbers in Texas and California reflect the distribution in those states. Seven local bar associations distributed the survey in California (San Francisco, Contra Costa County, Los Angeles County, Monterey County, San Diego County, Santa Barbara County, and Riverside County). The State Bar of Texas sent the survey link to a random active membership sample of 16,000.

8.  All surveys were completed electronically, with the exception of 7 completed in hard copy. Qualtrics, a survey software available through the University of Denver, was the survey platform utilized.

9.  A response was considered valid as long as the respondent provided answers for at least one of the substantive items and identified as holding either a position providing legal services or a position for which a J.D. was advantageous (or indicated a plan to return to such a position).

10.  In order to most accurately report the findings for each state, we defer to the respondent’s reported office location, rather than the state bar that distributed the survey link to the respondent, in classifying respondent state.

11.  Although there is no hard-and-fast rule, generally, an acceptable margin of error is 5%, with the ideal being 3% or lower.

Attorney respondents represent a multi-faceted and varied group on all measured dimensions, including demographics and practice-related characteristics. The sections below delve into each of these in detail.

Demographics

Gender. Men (59%) represented a larger proportion of the respondent group than did women (41%);(12) however, this is roughly consistent with the proportions of men and women who are licensed attorneys in the United States—65% and 35%, respectively.(13)

Figure 2: Gender (n = 18350)

Racial and Ethnic Background. At just over 90%, White non-Hispanic(14) attorneys made up the broad majority of the respondent pool. The second largest, but substantially smaller, groups were Hispanics and Black or African-American non-Hispanics, each of which represent 3% of respondents. About 2% indicated being multi-racial.(15) While it may seem that these numbers slant heavily toward Caucasian respondents, closer inspection reveals that they largely mirror the national numbers of attorneys in each racial and ethnic group. Indeed, the 2010 U.S. Census data shows that 88% of practicing attorneys are White non-Hispanics, while 5% are Black or African-American non-Hispanic and 4% are Hispanic.(16)

Figure 3: Racial and Ethnic Background (n = 16858)

Income. Attorney respondents tended to be in the mid-range with respect to annual income. One-third (33%) reported an income between $50,000 and $99,999, while just over one-quarter (26%) earned between $100,000 and $149,000. Smaller proportions of respondents reported an annual income of less than $50,000 (9%), $150,000 to $199,999 (13%), or more than $200,000 (19%).

Figure 4: Income (n = 17775)

Practice-Related Characteristics

Type of Position.(17) About two-thirds of respondents (65%) reported working in a position providing legal services, while only 10% reported working in a position for which a JD is advantageous. A sizeable minority (25%) reported working in a position both providing legal services and for which a JD is advantageous.

Figure 5: Type of Position (n = 24137)

Primary Work Setting. A majority (58%) of respondents identified private practice as their most recent primary work setting, with about one in five (19%) being solo practitioners and one in four (23%) working in firms with between two and ten attorneys. Attorney respondents were spread across a wide variety of additional practice settings, though no other setting represented more than 10% of respondents. These proportions generally align with national data on attorney practice settings, where private practice constitutes the largest percentage by a wide margin, followed by government practice (75% and 8%, respectively).(18)

Figure 6: Primary Work Settings (Collapsed) (n = 23080)

Figure 7: Primary Work Settings (n = 23080)

Type of Practice. Two-thirds (66%) of respondents worked in litigation, while 40% had a transactional practice and 22% had a regulatory practice. Just over one in ten (13%) attorney respondents indicated they did not currently practice law. Because attorneys may have more than one practice type, respondents were allowed to select all applicable options—thus, the reported percentages add up to more than 100%.

Area of Expertise. Attorney respondents were asked to select up to three areas of expertise from an extensive list of options. Criminal law (17%), general civil litigation (16%), and family law (15%) were the most common areas of expertise. Figure 8 below presents the proportion of respondents who selected each option.(19)

Figure 8: Areas of Expertise (n = 23837)

New Lawyers in the Workplace. Just under one-third (30%) reported having a role in hiring new lawyers, yet abroad majority (70%) did not. Similarly, just over a third (35%) of respondents reported having a role in supervisingnew lawyers, but the remaining two-thirds (65%) did not. A larger proportion—about half (49%)—reported working with new lawyers on substantive matters, committees, or other meaningful projects. Figure 9 provides more detail on respondents’ work with new lawyers. In terms of junior lawyer hiring practices, a little more than half (51%) of respondents reported that most junior lawyers hired in their workplace are hired as entry-level candidates, while about one-quarter (26%) reported that most junior lawyers were hired laterally after training at another firm or organization; the remaining quarter (23%) reported a split between hiring junior lawyers as entry-level candidates and lateral hires.(20)

Figure 9: Interactions with New Lawyers in the Workplace(21)

Office Location. In terms of developed environment, the vast majority (87%) of attorney respondents practiced in and around cities—half (50%) of respondents practiced in urban areas, while just over two-thirds (37%) practiced in suburban areas.(22) Substantially fewer (13%) attorney respondents had a rural practice. The most commonly reported practice locations were Portland, Oregon (3%), New York, New York (2%), and Seattle, Washington (2%).

Years in Practice and Law School Attended. Respondents ran the gamut in terms of number of years in practice.The shortest amount of time since law school graduation was one year and the longest was 74 years; the average was 21.7 years. Figure 10 below illustrates the distribution of respondent years since law school graduation. Just over one in four (29%) respondents attended a tier one law school; 8% attended a top 14 law school.(23) Almost one-third (30%) attended a tier two school, while about one-quarter (26%) attended a tier three law school. The remaining 15% attended a law school that was either unranked or not subject to ranking (e.g., foreign or unaccredited). The most commonly reported law schools from which respondents graduated were Thomas M. Cooley Law School (2%), Lewis and Clark College (2%), and Wayne State University (2%).

Figure 10: Years Since Law School Graduation (n = 23099)

Figure 11: Law School Tier (n = 18052)

Endnotes:

12.  This calculation, along with all others presented, includes only those who provided a response to the specific item—those who left the survey item blank or indicated prefer not to answer have been removed from the relevant calculation.

13.  Lawyer Demographics, Year 2015, Am. Bar Ass’n (2015), http://www.americanbar.org/content/dam/aba/administrative/market_research/lawyer-demographics-tables-2015.pdf.

14.  The U.S. Census treats Hispanic as an ethnic identity; Hispanics may identify as any race. In order to maintain consistency and comparability with U.S. Census data, we have chosen to report racial and ethnic background in an identical manner.

15.  The remaining 2% were either Asian, American Indian/Alaska Native, Pacific Islanders, or indicated Other.

16.  Lawyer Demographics, Year 2015, supra note 13. The ABA demographic statistics cite to the U.S. Census, which collects employmentstatistics including sex, race, and Hispanic origin by occupation. Industry and Occupation, U.S. Census Bureau, http://www.census.gov/people/io/ (last visited May 23, 2016).

17.  Calculations include respondents who currently work in such positions and those who expressed an intent to return to such positions.

18.  Lawyer Demographics, Year 2015, supra note 13. Note that there are fewer categories listed in the cited source than were presented in the Foundations for Practice survey.

19.  There was a small group of respondents (1.2%) who did not select an area of expertise.

20.  This calculation excludes respondents who indicated that the question was not applicable or that they were unsure.

21.  The presented values include solo practitioners, as it is conceivable that they could have contact with new lawyers in a manner relevant to the question. However, to ensure inclusion of these attorneys was not skewing the data, we ran the calculations excluding solo practitioners. The proportions were virtually identical, with the largest difference being that 5.1% more (53.9%) respondents indicated yes to the statement “I work with new lawyers on substantive matters, committees, or other meaningful projects.”

22.  To determine the developed environment, we used respondents’ city data combined with the GreatData Rural Urban Suburban Codes Database, containing zip codes and cities classified “based on three key factors: population density (people per square mile), distance from nearest city, and size of the nearest city (urban and suburban areas extend farther for larger cities).” Rural Urban Suburban Data, GreatData, http://greatdata.com/rural-urban-data (database on file with authors). If a city was not in the database, we used the following census population density statistics to designate the city as urban (3000+ persons per square mile), suburban (1000-3000 persons per square mile), or rural (fewer than 1000 persons per square mile).

23.  Reported ranks are drawn from the 2016 U.S. News and World Report rankings. U.S. News & World Report, Best Graduate Schools 2016 100-107 (2015).

This survey was a significant undertaking and resulted in a rich data set that we believe will advance research and action to improve legal education and to ensure that new lawyers are positioned to launch successful careers, enter the profession, and serve the needs of their clients. We partnered with many people and organizations across the country to administer the survey and are grateful to the bar organizations, courts, lawyers, judges, and legal educators who made this possible.