I’m freshly returned from the American Association of Law Libraries annual meeting, full of ideas and questions. I’ll touch on the innovation panel on which I was a judge later but first I want to return to value. ROI. Are we still equating marketing and communication with value measurements?

 

If – and I’m not sure this is entirely settled – law libraries only exist so long as they can justify their existence, I’m still not seeing good examples of how they explain the last mile. We collect usage statistics and foot fall metrics. We analyze the data. But we don’t seem to be able to attach a monetary value to it. That’s important when funders have $1 and need to decide to give it to a law library, or volunteer fire department, or ambulance service.

A tongue-in-cheek law library take on the South Park Underpants Gnome meme

And I don’t mean any disrepect to the many people who I’ve seen talk about their law library ROI and ended up talking about their marketing streams. That’s an important component to promoting library services in order to generate whatever return there could be.

I’m using a monetary lens not because it’s the most important. It’s certainly not the only one but the soft comparisons – goodwill, charity – are harder to quantify. When justifying a law library operation to funders, understanding return on investment in terms of dollar value is probably the most powerful comparison.

I certainly haven’t figured it out myself. I’ve gathered the metrics, I’ve told the stories and used a variety of visualizations. It’s the final step that’s missing.

My notes from the AALL 2019 conference

You may want to read the AALL Defining ROI whitepaper from 2016. The article from Colleen Cable stands out to me as the most universally useful on a wide variety of metrics. But much of the other approaches deal with what I have done myself: measuring early and often, and lots of communication.

We Are Missing Numbers at the End

We have plenty of numbers at one end of our ROI assessment. We measure users in our databases, books moved off shelves, foot traffic, emails sent, questions answered, documents delivered, books loaned through ILL or other services.

What about the numbers at the end of the assessment? That’s where the “return” in ROI lives.

One of the best examples I saw was a reference to a Minnesota survey cited by Perkins Coie’s Amy Eaton at AALL 2019. In essence, the argument was this:

Law library value = the delta between the cost incurred by a timekeeper to do legal research and the cost incurred by library staff to do the same research

This is intriguing because it is something that would work in any library. You essentially measure (or ask) about the work shifted from the person who needs it to the person who delivers it. It’s aligned with lots of other work in law firms: partners delegate to associates, lawyers delegate to paralegals, etc.

It’s still a bit vague, though. Let’s say the cost of an associate doing a project is $1,000 but a librarian can do it for $350. Librarians should be faster and should cost the firm less. We’re making the assumption that the lawyer and the librarian would have taken the same amount of time.

What about an instance where a lawyer needs 10 cases pulled and updated (whatever Shepardizing is these days)? The lawyer asks an secretary to do this, rather than a librarian. The savings may be greater, but the value the library is providing is selection and access in making the resource available. How does the library translate that usage into value?

It also assumes that a lawyer will engage in a comparable amount of research. If we think about legal research as hitting a good enough plateau, it may be that a lawyer is willing for a law librarian to spend far more time on a project than she would herself. Or she might feel that the law librarian is no more efficient than the lawyer, and so it is merely a difference in billable costs that creates a savings.

One benefit to law firms is that they can, in theory, trace library impact across the lifecycle of a matter. Two things occurred to me that I assume are feasible (if, ultimately, pointless):

  1. Law firms can often assess the expected cost of a client’s matter, based on previous experiences. That’s how you get to alternative fee arrangements. At year end, I wonder if a law firm library could look at the matters that had library billable time (whether or not the client was billed for it) versus the matters that did not. And is there any correlation between matters with library time being closer to (or lower than) average matter costs than those that didn’t use library services.
  2. Legal research in a matter is not always discrete, knotted with a silver thread to the final resolution for the client. But if you have a matter that uses library support, and you’ve made a calculation of timekeeper shifted value, isn’t that represented at the end balance? If a matter generates $50,000 in revenue and the actual cost to the firm is only $40,000, does the shifted value also now have the same 125% multiplier as the other costs do? I know I’m making a hash of this, but essentially, is there a better way to tie the cost shifting to the revenue result.

The arc of realizing the information may be too long to be useful but I think it would be worth following. Every matter is eventually closed. A closed file sets off triggers in an organization, whether it’s for records management teams or other reasons. It may be a good time to discern whether, over a matter’s lifetime, the library’s involvement had any impact. And whether that impact is truly an apple-to-apple cost shift.

Getting Feedback

This is squishier outside law firms but not, I think, impossible. The timekeeper value shift is the same for a single mother who can’t take time off work, or a rural researcher who lives far from the library. Their time has value too.

We might try immediate feedback by asking two questions:

  • “how did the law library’s help save you time?” and
  • “what is the value of that time to you?”

The feedback needs to be sought when the interaction happens and it has to be easy. It’d be like a customer service email that asks you to rate the service in stars, and then forwards you to a few-question survey to flesh out your response.

A hotel followup survey. When you click a number, you give feedback and are sent to an external survey. There is also an expiration note to suggest some (false) urgency.

Another option for courthouse law libraries would be to see if they can capture efficiency savings realized elsewhere in the system. It requires a similar approach to a closed law firm matter file. And it would require coordination with a judges chamber and the clerk of courts.

Maybe It Doesn’t Matter

There seems to be an acceptance between funders and law libraries that we can’t actually prove our value. It may be that we all understand that law library value is an intuitive thing. The inability to ascribe a dollar value to law library operations is not always determinative in decisions to keep or close a law library. Unfortunately, without a financial value, law libraries are in more uncertain position..