Cost is the primary issue that often lies at the heart of e-discovery disputes, and since 2015, Rule 26 of the Federal Rules of Civil Procedure (FRCP) was amended to require that the scope of discovery in litigation would be proportional to the value and needs of the case.
This article explores the relationship between the FRCP proportionality mandate and the cost and efficiency of different review techniques and technologies, including linear review, keyword search, TAR 1.0, and TAR 2.0—particularly in light of the Sedona Conference Proportionality Commentary, Principle 6 (“technologies to reduce cost and burden should be considered in the proportionality analysis").
Judge Andrew Peck foreshadowed this issue in Hyles v. City of New York, No. 10 Civ. 3119 (S.D.N.Y. Aug. 1, 2016). In Hyles, the plaintiff wanted Judge Peck to force the city to use technology-assisted review (TAR), because TAR would simply be more efficient and more effective than their proposed approach of using keywords.
Although Judge Peck determined that the city could not be forced to use TAR “yet”, he noted that “[t]here may come a time when TAR is so widely used that it might be unreasonable for a party to decline to use TAR”. In other words, there may well come a time when a court sees fit to require a party to use TAR to properly discharge its discovery obligations.
Magistrate Judge Katharine H. Parker recently went a step further in Winfield v. City of New York (S.D.N.Y., Nov. 27, 2017).
In Winfield, the city was using keyword search and linear review techniques, and their document production was seriously languishing. Dissatisfied, the plaintiffs lodged numerous complaints surrounding the pace of discovery and document review. Ultimately, “th[e] Court directed the City to… begin using Technology Assisted Review (“TAR”) software (also commonly referred to as “predictive coding”) to hasten the identification, review, and production of documents responsive to Plaintiffs’ document requests”.
More Judges Recognizing E-Discovery Need
Although neither case directly implicated proportionality, Hyles, and more particularly Winfield, signal the judiciary’s increasing recognition of the need to consider the impact of technology when resolving discovery disputes, consistent with Principle 6 of the Sedona Conference Commentary.
And the progression from Hyles to Winfield suggests that judges are beginning to look much more critically at the technological approach being used by the parties to discharge their discovery obligations—encouraging remedial changes where it is appropriate to do so.
As courts continue to delve more deeply, the inquiry will inevitably reach the interplay between technology and proportionality. When that happens, courts will have to consider whether the producing party’s approach to review and production is just so inherently and unnecessarily costly or inefficient that a proportionality objection will not be entertained to limit discovery, particularly when less expensive and more efficient alternatives are readily available.
Cost Vs. Efficiency
Take, for example, the decision to use a single-pass TAR 1.0 simple learning approach to a technology-assisted review, rather than a TAR 2.0 continuous active learning technique—which studies have generally shown to be more efficient.
Contemporary ESI protocols anticipate recall levels (the fraction of responsive documents located in discovery) in the range of 70 percent to 80 percent. It is not uncommon, however, for TAR 1.0 technologies to stabilize before achieving 70 percent recall.
In practice, that means that no amount of additional review and training will improve the predictive model, and recall will likely never reach 70 percent. From a proportionality perspective, that suggests that additional review might be inappropriate, since the cost will likely outweigh the effective benefit. But, in reality, TAR 2.0 is a viable alternative technology that would likely achieve the anticipated recall levels, and do so even more efficiently.
In the same vein, certain TAR techniques rely heavily on human involvement and, therefore, are relatively expensive. And the larger the collection, the greater the expense. When those techniques are used, there is often a concerted effort to minimize the collection subject to TAR, typically through the application of traditional keyword culling.
Proportionality concerns might suggest that culling is appropriate, given the added expense of applying TAR to the entire document population. However, studies have shown that keyword culling is not very effective, such that a significant fraction of the responsive documents never pass through to the TAR process. And again, in reality, there are alternative algorithmic TAR techniques that would be relatively equally effective, and at far less cost.
Both of these examples illustrate the conundrum facing the courts as they seek to refine the relationship between proportionality and technology, particularly in light of the Sedona Conference Commentary.
Should a proportionality objection be resolved without reference to technology alternatives, or will a litigant be obligated to first demonstrate that review protocols and technologies have been reasonably optimized? Eventually, that question will need to be answered. But, much as Judge Peck observed in Hyles, we’re not there … yet.
Thomas Gricks is director of data analytics for Catalyst (part of OpenText). He advises corporations and law firms on best practices for applying TAR technology.
John Pappas is director at Catalyst (part of OpenText), which designs, builds and runs platforms for complex e-discovery, regulatory investigations and compliance.