Going Public with Innovation: What the ETL Survey Aims to Discover

In the first blog for Educating Tomorrow’s Lawyers, Bill Sullivan argued that legal education is more than meritocracy – that while academic merit is essential in the training of lawyers as professionals it is not enough. It is necessary, but not alone sufficient for professional excellence. This is the basic argument of Educating Lawyers, the 2007 report of the Carnegie Foundation for the Advancement of Teaching, of which Bill was the primary author. In addition to a mastery of the necessary formal academic knowledge and the technique of legal reasoning, Educating Lawyers says training for professional excellence also requires two more things: learning to practice by bridging the gap between the academic and the practical world of lawyers’ work; and initiating students, as Bill put it, “into the social roles, ethical standards, and responsibilities that embody the fundamental purposes of the profession of law.”

That 2007 report makes a powerful argument about the need to broaden legal education and it comes in a long line of previous critiques of legal education, some going back to the early 20th century. Moving forward in this vein, Educating Tomorrow’s Lawyers has been formed to foster critical and creative thinking and activity with regard to legal education in light of that powerful argument. A key prerequisite for doing so is having a reasonably accurate understanding of what is happening in legal education, and this is my theme.

In preparing Educating Lawyers, Bill and his Carnegie colleagues conducted in-depth analyses – including on-site visits – of programs at 16 very different law schools focusing on teaching and curriculum. Their work provides an unparalleled, critical picture of legal education at the end of the 20th century. It has been a decade since that work was done and it is time to see what the geography of legal education looks like now as we move forward. With this in mind, Bill and I designed an online survey (administered by the University of Denver’s Butler Institute) built around the key features of Educating Lawyers’ recommendations, with a special emphasis on integrative approaches that link doctrine and practical experience. We sent the survey to the deans of every ABA accredited law school in the United States and to the deans of the Anglophone Canadian law schools with common law programs. As of early June over 110 deans have completed or started the survey, and we hope to receive more.

The survey asks about substantial changes or new initiatives since 2001. We chose 2001 as the starting point in order to capture changes or initiatives that pre-dated the publication of Educating Lawyers. We assume that at least some schools were experimenting with innovative teaching and curricular matters before the publication of Educating Lawyers (in fact, the report describes noteworthy ones that were uncovered in the research). Our interest is in movement toward the kinds of reforms Educating Lawyers proposes and whether that movement is sustained and institutionalized. The complexities involved in bringing about large-scale change in legal education counsel against making bold causal claims, but developing a clearer picture of innovation in legal education today provides a resource of previously unavailable knowledge that will extend, and if necessary correct, the findings of the 2007 Carnegie report.

The survey focuses on changes and initiatives at the institutional level in five areas: curriculum; faculty development related to teaching and learning; faculty professional activity related to teaching and learning; assessment of student leaning, experience, and performance; and assessment of the success of a law school’s educational program. Focusing on the institutional level is important as a starting point because without change and support at this level, the efforts of individual teachers in their classes to innovate may be for naught. Sustained reform in legal education requires an institutional commitment, and we think such a commitment will be reflected in what a school does in each of these areas.

The survey asks each dean if his/her school has been involved with any substantial changes or new initiatives in each of these areas since 2001. If a respondent answers affirmatively, the survey then asks a series of questions about more specific changes within that area. The deans are asked whether there has been a change involving that more specific issue, the year in which consideration of that change began (2001 to 2010), and where they are in the process of implementing that specific change (from mere consideration to full implementation). Again, we want to get a good sense of what the geography of legal education looks like now – including those still in process as well as those already in place.

For example, take the first broad area in the survey—curriculum. If a dean answers affirmatively to the threshold question about substantial changes or new initiatives since 2001, the survey takes the dean to a new page with 11 more specific changes within the area of curriculum. Those possible changes range from curricular changes for each of the three years of matriculation to changes involving clinics and certificate programs to changes involving lawyering or other skills courses to changes involving professional identity or ethics courses. Perhaps most importantly in light of Educating Lawyers’ recommendations, the survey specifically asks about changes involving integrative approaches that link doctrine and practical experience. For each item to which the dean answers affirmatively, he/she is then asked the year in which consideration of that item began and how far along toward implementation they are.

The second broad area is faculty development related to teaching and learning. Here our interest is in institutional support for innovation in teaching. Looking for support for faculty development on the part of the school makes a public statement about the seriousness of a school’s commitment. The survey asks about organized workshops or conferences at a school relating to teaching and learning generally, and about workshops or conferences related to integrative approaches specifically. It also asks about faulty grants or support to aid in the development of innovative approaches to teaching generally and integrative approaches specifically.

The third broad area is faculty professional activity related to teaching and learning. Here our interest is in whether such activity is valued and rewarded since without appropriate incentives it is hard to see how reform can succeed. The survey asks about the role of professional activity related to teaching and learning in faculty hiring decisions, in decisions about promotion and tenure, and other merit decisions. Each of these questions reveals aspects of a school’s internal incentive structures and what these structures effectively foster or inhibit.

The remaining two areas of the survey deal with assessment – student and school, respectively. Implementing change successfully requires assessment – evaluating whether a change actually makes any difference. With regard to student assessment the survey asks about new ways (other than the traditional exclusive reliance on the final exam) of assessing student performance in 1st and in 2nd and 3rd year classes as well as in clinic or other experiential programs. The survey asks specifically about assessment within integrative classes or programs. It also asks about any changes or initiatives in assessing the law student experience, including the use of the Law School Survey of Student Engagement. With regard to school assessment the survey asks about new initiatives directed at assessing post-graduate success that are, in turn, linked to an assessment of the school’s educational program. This includes the development of feedback mechanisms such as surveys of employers or potential employers of the schools graduates.

Finally, for each of the five broad areas the survey also includes an open-ended question asking each dean to describe briefly one or two changes or initiatives in that area that he/she sees as the most distinctive or important for his/her school. The idea is to capture at least some of the particular things being done that more general questions will necessarily miss. Responses to these questions are giving us a better flavor of what is being done. They also provide an important starting point for the subsequent research efforts we envision. We intend to dig more deeply into each of these five broad areas by following up the responses to our questions with those schools that indicated they are making substantial investments of creativity and educational resources toward improving legal education.