In 2011, Maura Grossman and Gordon Cormack published their landmark study of technology assisted review, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient than Exhaustive Manual Review. Discussing the general state of opinion at that time, they explained, “E-discovery processes that use automated tools to prioritize and select documents for review are typically regarded as potential cost-savers – but inferior alternatives – to exhaustive manual review, in which a cadre of reviewers assesses every document for responsiveness to a production request, and for privilege.”
But the results of their work suggested that the common wisdom was, in fact, wrong. They continued—in the very next sentence in the abstract—to state, “This Article offers evidence that such technology-assisted processes, while indeed more efficient, can also yield results superior to those of exhaustive manual review, as measured by recall and precision, as well as F1, a summary measure combining both recall and precision.” In the intervening years (almost an eternity in the rapidly evolving field of e-discovery technology), time has marched on and the lesson is clear: the machines have won. (Interestingly enough, while the machines are clearly better at review, a variety of factors have combined to make the economic case for TAR less clear-cut.)
Potential Shortcomings of Predictive Coding
Predictive coding is the form of artificial intelligence that has had the greatest impact on e-discovery thus far. To quote our The Basics of E-Discovery Guide, Predictive coding is a machine learning process that uses software to take keyword searches and logic, entered by people, for the purpose of finding responsive documents, and applies it to much larger datasets to reduce the number of irrelevant and non-responsive documents that need to be reviewed manually.
But if one digs into the reasons why legal teams may not get the full benefit of predictive coding, unsurprisingly a lot of it has to do with the fact that predictive coding technology, despite its superiority at document review, remains dependent on its human masters in the creation of the seed sets needed as a precondition to actual document review. Users who want to get the most out of predictive coding should start by creating good sample sets and defining useful parameters.
But that’s not always the case. All too often, users treat the technology like it’s magic. Rather than learn how it works, Keeling explains that many users think, “I don’t need to exercise any kind of discretion as far as what goes into it, because it’s magic and it’s just going to figure it all out.”
A Way Out of the Human-Bias Trap?
One way that emerging e-discovery technology can address this problem is by eliminating the requirement for seed sets prior to review. Just because a person is capable of performing effective review does not mean they are equally capable of training an AI tool to perform review. Are they constructing a deep enough and varied enough pool of documents to enable the tool to make recommendations on all sorts of ESI pertinent to a task?
Generalizing from this particular dilemma, it’s safe to say that people, while quite capable of performing many highly technical tasks effectively, may not have the ability to analyze the tasks equally effectively—a variation on the maxim, “Those who can, do; those who cannot; teach.” (I much prefer David Foster Wallace’s articulation from his essay, How Tracy Austin Broke My Heart, “It may well be that we spectators, who are not divinely gifted as athletes, are the only ones able truly to see, articulate, and animate the experience of the gift [genius] we are denied.”)
So if this notion has some truth in it—that those with a genius talent are not necessarily able to discuss (or teach) it—perhaps the better way forward for artificial intelligence is not to operate on rules dictated by humans, but to learn them on its own, through observation of the action to be emulated.
Luckily for e-discovery professionals, that is now an option. Exterro Smart Labeling, the industry’s most advanced use of Artificial Intelligence (AI) in the Exterro Review Platform, leverages the latest Deep Learning technology innovations to help guide reviewers by constantly analyzing and understanding the context of their labeling decisions, and suggesting/queueing the most relevant unreviewed documents, saving significant time and cost.
“Exterro Smart Labeling is built upon the latest innovations in Deep Learning, Natural Language Processing, compute intensive hardware processing and other related architecture approaches, delivering far superior results and experiences than previous machine or continuous active learning approaches,” said AJ Samuel, Chief Product Officer at Exterro.
Exterro’s Smart Labeling algorithms interpret language in ways that mimic how the human brain functions. Rather than processing exhaustive sets of rules, it recognizes key terms and understands context — such as the meaning of words, sentence structures, and word choice patterns. As reviewers perform their jobs, the Smart Labeling algorithms learn and refine their understanding of which documents are relevant and apply labels to unreviewed documents with a corresponding confidence score, helping reviewers view the most highly likely relevant documents sooner—saving considerable time and expense. The approach requires no seed sets, no additional training, and no changes to current review processes, as the algorithms work in the background while reviewers complete their tasks.
If you’re interested in finding out more, sign up for a demo on the Exterro Smart Labeling product page today.