How do we evaluate the quality and value of legal services? For example, if we compare two proposed contracts for a commercial agreement, how do we determine which contract is of higher quality? How do we determine the total value produced by the process of drafting, negotiating, and finalizing each contract? Would our answers change if some or all of the services are produced by a software application? If a software application is used, how would we evaluate the quality of any training data inputs, the development process, and the outputs of the software application? Would our assessment of the quality and value of the software application change if the software application is used to serve individuals who would otherwise go without a lawyer?
These are just some of the questions that I discuss in this draft book chapter, Evaluating Legal Services: The Need for a Quality Movement and Standard Measures of Quality and Value, the final version of which will be available in the Research Handbook on Big Data Law edited by Dr. Roland Vogl, forthcoming 2020, Edward Elgar Publishing Ltd. I’ve included the abstract for my chapter below.
With this chapter, I aim to demonstrate the need for a quality movement and standard measures of quality and value and highlight some of the research and resources. My goal is to catalyze debate, rigorous research, and sustained action. If we do not undertake this work, we risk squandering abundant opportunities to improve legal services, legal systems, justice, and the law itself.
I welcome input from everyone working on or interested in these topics. I’ve tried to identify relevant literature and initiatives. Please let me know what I’ve missed. I’ve also made the case for why we need a quality movement and measures of quality and value. Is there a better case to be made? Please download my draft book chapter and let me know what you think.
Finally, thank you to Northwestern Pritzker School of Law student Research Assistants Mona Kalantar, Cheron Z. Mims, and Richard “Hitch” Thomson for their contributions to this project!
Abstract – Evaluating Legal Services: The Need for a Quality Movement and Standard Measures of Quality and Value
The legal industry has not undergone a quality movement and lacks standard measures of legal-services quality and value. Whereas medicine long ago embraced evidence-based practice and empiricism, law muddles along, deferring to lawyers’ untested, loosely conceived, normative standards for practice. As a result, existing data about legal-services delivery is of questionable quality, and we lack standard metrics to evaluate this data and any applications developed with it. The lack of empirical rigor in legal services threatens progress in data analytics and artificial intelligence, which require high-quality input and output data. Additionally, the failure to undertake a quality movement in law contributes to numerous legal industry problems, including inadequate access to legal services and justice, a lack of diversity, and work-life imbalances. This article discusses the need for a quality movement (focused on standard work, error detection, peer review, performance measurement, and continuous improvement) and standards for legal-services quality and value. The article discusses output, process, and input data and metrics for measuring quality and value. The article includes summaries of multiple holistic models for measuring legal-services value, including from Noel Semple, Rebecca Sandefur and Thomas Clarke for “roles beyond lawyers,” and Paul Lippe for contracts. The article also identifies several initiatives contributing to the development of quality and value metrics. Finally, the article briefly summarizes the stakeholder benefits of a quality movement and standard metrics for legal-services quality and value.