eDiscovery is a signal-to-noise problem. In an era of massive data growth, legal professionals are faced with a deafening amount of largely meaningless data, data that threatens to drown out the information that matters. The goal of discovery technology is to empower lawyers to better sift through that data, to separate the noise from the signal, so that the work of discovery can get done faster, more efficiently, and (ultimately) more affordably.
Yet when that technology is prohibitively expensive, the benefits it brings can be significantly undermined, as a recent discovery dispute out of the Northern District of Illinois shows, with the parties duking it out over the cost and capabilities of a TAR-powered review process.
That dispute, County of Cook v. Bank of America Corp., No 14 Cv. 2280 (N.D. Ill. Oct. 22, 2019), involves litigation between Cook County, home to Chicago and its immediate suburbs, and several of the nation’s largest lenders over allegedly discriminatory mortgage practices that lead to the 2007 subprime mortgage crisis.
Cook County’s Predatory Lending Suit
Before we get into the discovery dispute, first some quick background. (Those of you who are already familiar with The Big Short can skip ahead.)
On September 25, 2006, home prices fell for the first time in over a decade. That dip in housing prices soon turned into a freefall, marking the beginning of the foreclosure crisis, as homeowners, many with subprime and interest-only loans, found themselves unable to make mortgage payments and unable to sell their homes. That foreclosure crisis kicked off a much larger financial crisis, helping spawn the Great Recession, whose effects are still being felt today.
By the time the market had fully recovered, over 7 million homes had been foreclosed and household net worth had dropped by $11.5 trillion.
Here’s where Cook County comes into the picture. In 2014, the County sued Bank of America and its subsidiaries, including Countrywide and Merrill Lynch, accusing them of predatory lending practices in violation of the Fair Housing Act. Five years later, the suit is still in the document discovery stage.
It’s Vendor v. Vendor in a Fight Over Whether TAR Can Handle 24 Custodians
Recently, the County sought to compel discovery from 24 additional custodians—a significantly narrower request than the original 785 custodians they had proposed.
The bank defendants argued, as is to be expected, that additional discovery would be disproportional. Under Federal Rule of Civil Procedure 26(b)(1), proportionality in the scope of discovery is determined by reference to six factors:
- The importance of the issues at stake
- The amount in controversy
- The parties’ relative access to relevant information
- The parties’ resources
- The importance of discovery in resolving the issues, and
- Whether the burden or expense of the proposed discovery outweighs its likely benefit.
Adding more custodians would significantly drive up costs and slow the review, according to the banks.
Both sides trotted out their discovery vendors to argue what was, or wasn’t, possible with their TAR process—and even what TAR process they were using.
On Bank of America’s side, the vendor estimated that the new custodians would provide 1.8 terabytes of data, requiring two to three weeks to process alone. By “degrading” the percentage of responsive documents, the new data would require “more review to find the responsive documents among the larger data corpus of non-responsive documents.” That review would take, the vendor estimated, six months. All told, Bank of America estimated the processing and review of data from the new custodians to cost more than $750,000.
Cook County’s own TAR vendor disagreed. Concerns that a higher ratio of responsive to nonresponsive documents (aka lowering the “richness” of the document corpus) would slow review were misplaced, the vendor claimed—characteristic of “TAR 1.0” rather than the “TAR 2.0” the bank said it was using. Further, there seemed to be little insight into how that TAR process worked, such as the projected richness, confidence levels, or even review rates. Processing should take days at best, the vendor claimed, and review costs should be negligible. The bank’s faulty assumptions over data size and review time “materially bloat the resulting cost and time estimates,” the vendor claimed.
TAR Battle Highlights Pitfalls of Prohibitively Expensive Technology
In the end, the court sided with the bank.
The technology available to the producing party, despite that technology’s promises, was simply too expensive and burdensome to render additional discovery proportional.
Indeed, according to the defendants, their TAR-powered review had already cost $1.3 million and required 36 doc review attorneys working full time for three months just to get through their initial 400,000 documents.
From the court:
The Court disagrees that TAR eliminates Judge Rowland's expressed concerns about the burden of ESI discovery in this case. Despite using TAR to target likely responsive documents, Defendants have reviewed 400,000 ESI documents to date for the 38 Court-ordered custodians. According to Defendants, "[si]nce mid-July 2019, some 36 attorneys have been reviewing documents collected from the Court-ordered custodians full time." Moreover, the charges by Defendants' ESI vendor for document processing, review, and production are projected to exceed $1,300,000 and this figure does not include Defendants' own personnel costs in collecting the documents or outside counsel's costs and privilege review work.
For those keeping track, 400,000 for 36 reviewers is approximately 11,000 documents per attorney. At a linear review rate of 50 documents an hour, that would require 222 hours of review time per lawyer, or five and a half full, 40-hour weeks—about half as long as the three months it took for the banks to go through the documents here, though we can assume other factors, such as the complexity of their TAR approach, or the complexity of the data and requirements for second- and third-pass review were also at play.
These numbers undermine any suggestion that Defendants' use of TAR to aid in their ESI production affects Judge Rowland's proportionality basis for denying the County's request for ESI from the custodians at issue here.
There are, of course, alternative outcomes to technology-focused proportionality arguments. A few months ago, for example, Logikcull published a case study on Slack data in litigation. In that case study, New York-based litigator David Slarskey was able to overcome objections to the production of Slack data by showing that, in fact, using Slack data in eDiscovery was not burdensome when the right technology was available.
But the County of Cook decision is an important reminder that technology alone isn’t enough to solve the problem of eDiscovery—particularly when that technology costs over millions of dollars to use.
This post has been adopted and slightly expanded from a recent article on why technological improvements alone, without drastic changes to eDiscovery pricing models, can't solve the burdens of discovery, published October 31st on the Logikcull Blog.
