Table of Contents >> Show >> Hide
- What Happened: The Case That Turned Discovery Into a Drama Series
- What Is TAR (Technology-Assisted Review), in Plain English?
- What the Court Endorsed: A TAR-Forward ESI Protocol Blueprint
- 1) Statistically valid random sampling first
- 2) TAR to define search parameters across the full collection
- 3) A “95% accuracy rate” target (and what to do with that in real life)
- 4) Cost sharing: the quickest way to make everyone suddenly “efficient”
- 5) Native production with metadata
- 6) Mutual inspection and no artificial limits on how many lawyers can review
- 7) The unusual twist: “clawback” for irrelevant documents
- 8) Counsel affidavits: “You’re officers of the courtact like it”
- Why This Matters: TAR Is No Longer “Cutting Edge,” It’s Judicially Normal
- Practical Takeaways: How to Use This Decision Without Needing a Magistrate Judge to Raise You
- 1) Build an ESI protocol early (and treat it like a living document)
- 2) If you use TAR, document your workflow like you’ll have to explain it (because you might)
- 3) Consider cost sharing or cost-aware proposals before someone else proposes them for you
- 4) Treat metadata as part of usability, not an optional accessory
- 5) Plan for mistakes: build in correction, clawback, and cleanup steps
- Common TAR Myths (That Keep Showing Up Like Spam Emails)
- FAQ: Quick Answers for Busy Humans
- Experiences From the Trenches: 500-ish Words of “Yep, This Happens”
- Conclusion: TAR Wins When Cooperation Loses
Discovery is supposed to be the part of a lawsuit where everyone calmly exchanges information like civilized adults.
And then… reality shows up with a fog machine and a folding chair. In one federal case, the court opened its order
with a line that sounds like it belongs on a movie poster:
“Today we write the next chapter in this litigation, a case which threatens to become an epic of dysfunctional discovery.”
The punchline (for the rest of us, anyway) is that the judge didn’t just scold the parties. The court rolled up its sleeves,
wrote an ESI protocol framework, and endorsed Technology-Assisted Review (TAR)a.k.a. predictive codingas the most efficient
way forward. If you work anywhere near eDiscovery, this is one of those “clip it, save it, send it to your team” moments.
This article breaks down what happened, what the court actually endorsed, why TAR keeps getting judicial love, and what you can
do to avoid starring in your own “epic of dysfunctional discovery.” (Spoiler: start cooperating before the judge has to write
your discovery plan like a substitute teacher taking away recess.)
What Happened: The Case That Turned Discovery Into a Drama Series
The setup: high stakes, high conflict, and a high document count
In Everlast Roofing, Inc. v. Wilson (Middle District of Pennsylvania), the lawsuit involved claims like breach of contract,
tortious interference, unfair competition, and misappropriation of trade secrets. The plaintiff sought more than
$24 million in damagesso discovery wasn’t a side quest. It was the main storyline.
The problem: the parties couldn’t agree on basic ESI search terms and sampling. One side’s search approach suggested roughly
100,000+ potentially responsive records, while the other side’s approach yielded around 2,000.
That’s not a “reasonable disagreement.” That’s a canyon with a gift shop.
The court’s patience ran into a hard proportionality limit
The court had previously ordered counsel and their ESI vendors to meet in person and implement a collaborative, data-driven
sample testing strategy. The parties returned with dueling narratives that didn’t even agree on what was said in the meeting.
The judge’s message was blunt: counsel “can, should, and must do better.”
When cooperation fails, courts don’t just shrugthey manage discovery under the Federal Rules. And in this case, the judge
decided that the parties’ “intransigence” left the court no choice but to set the direction.
What Is TAR (Technology-Assisted Review), in Plain English?
TAR is a workflow that uses software to help classify documents as likely responsive or not, based on examples coded by
knowledgeable reviewers. Think of it as teaching a system what “relevant” looks like in your case, then letting it apply
that learning across a large pile of ESIwhile you validate results with sampling and quality controls.
TAR is not a robot lawyer
Courts tend to like TAR because it’s not magicit’s measurable. TAR typically comes with:
- Training sets (documents humans review to teach the system),
- Iterative learning (the system improves as reviewers code more examples),
- Validation (sampling and metrics to test how well the process performed), and
- Defensible documentation (so you can explain what you did and why it was reasonable).
And that wordreasonablematters. TAR is usually evaluated the same way keyword searches are evaluated: did you take a
reasonable, proportional approach to finding responsive information?
What the Court Endorsed: A TAR-Forward ESI Protocol Blueprint
The most important part of this decision is not just “TAR is allowed.” Courts have been saying that for years. The standout
is how the judge framed the protocol around sampling + predictive analytics + agreed validation targets, while also
addressing the very human problems that blow up discovery (cost fights, metadata arguments, who reviews what, and what happens
when irrelevant documents slip into production).
1) Statistically valid random sampling first
One side wanted a big random sample (like 1/10 or 1/20 of the dataset). The other side proposed a more analytical approach:
pull a smaller statistically valid random sample, review it, and use it to design smarter searching and review.
The court leaned toward the data-driven approach: start with statistical validity, then use the sample to shape the next steps.
This is discovery with a dashboard instead of a dartboard.
2) TAR to define search parameters across the full collection
The court described TAR as software that learns to distinguish responsive from non-responsive documents based on coding decisions
by knowledgeable reviewers on a subset of the collectionthen uses that learning to establish search parameters for the rest.
The key takeaway is that the court treated TAR as the most efficient and effective way to proceed under the circumstances.
3) A “95% accuracy rate” target (and what to do with that in real life)
The order referenced using the smaller sample to define parameters aimed at achieving a 95% accuracy rate.
In TAR land, “accuracy” can mean different things depending on context (precision, recall, confidence levels, or validation outcomes).
Practically, this signals that the court wanted the parties to:
- Agree on a validation method (often sampling-based),
- Define what success looks like (metrics and thresholds), and
- Use that agreed framework to guide review decisions instead of endless arguing.
Translation: stop debating feelings; start measuring results.
4) Cost sharing: the quickest way to make everyone suddenly “efficient”
ESI discovery can be expensive, and the court recognized that cost incentives can shape (or warp) each side’s position.
So the judge ordered cost sharing for future ESI discovery as part of the protocol.
Cost sharing does two things: it discourages “scorched earth” requests and discourages “we found nothing” responsesbecause both
sides now have skin in the game. When everyone pays, everyone reads the invoice line items.
5) Native production with metadata
The defendants wanted production in native format with accompanying metadata. With cost sharing in place (reducing one party’s
burden argument), the court agreed and ordered that approach.
Metadata disputes often sound boring until you realize metadata can show who created a document, when it was modified, who it
was sent to, and whether your “final_final_v7_reallyfinal.docx” story is… aspirational. Courts increasingly treat metadata as
part of producing ESI in a useful form, not a fancy bonus.
6) Mutual inspection and no artificial limits on how many lawyers can review
The parties fought over which counsel could review samples and how many attorneys could be involved. The court declined to cap
the number of lawyers and emphasized mutual inspection of samples so the parties can evaluate relevance and refine search
parameters.
This is a quiet but important point: TAR works best when the workflow is collaborative enough to be trusted, even if the parties
disagree about the case merits. You don’t need friendship. You need a process that can survive scrutiny.
7) The unusual twist: “clawback” for irrelevant documents
Clawback provisions usually show up in privilege contexts. Here, the court required the protocol to provide for the prompt
return or destruction of irrelevant information produced as the parties refine search parameters.
That’s a practical recognition of how iterative discovery really works: you refine, you test, you adjustand sometimes
irrelevant material slips through. Rather than treat that as a moral failing, the court built cleanup into the protocol.
8) Counsel affidavits: “You’re officers of the courtact like it”
One side wanted affidavits about the process. The court generally said affidavits weren’t necessary because attorneys already
have duties of candorthough it left room for affidavits if a party insists, in which case everyone would be subject to the
same requirement.
The subtext is loud: the judge didn’t want discovery turning into “discovery about discovery” unless truly needed.
Why This Matters: TAR Is No Longer “Cutting Edge,” It’s Judicially Normal
The Everlast Roofing decision fits into a broader U.S. trend: courts repeatedly recognize TAR as an acceptable (often preferable)
way to handle large document sets, especially when proportionality and cost are real concerns.
From early predictive coding to modern TAR playbooks
More than a decade ago, courts began approving predictive coding protocols. Over time, case law expanded from “is TAR allowed?”
to “how do we validate it, document it, and fight about it less?”
Modern guidance (including well-known judicial discussions and eDiscovery best-practice publications) increasingly frames TAR as
a tool that should be judged by the same standard as other search methods: reasonableness, not perfection.
The Sedona Principles vibe: cooperation is not optional, it’s table stakes
Courts often reference eDiscovery best practices that emphasize cooperation, transparency where appropriate, and proportionality.
The Everlast Roofing order practically begged the parties to stop acting like discovery is a competitive eating contest.
Rule 26 proportionality is the grown-up in the room
Federal discovery is not “anything that might be interesting.” It’s relevant, nonprivileged information that is also
proportional to the needs of the case. When parties can’t self-regulate, judges will enforce proportionality by shaping
scope, format, and methodexactly what happened here.
Practical Takeaways: How to Use This Decision Without Needing a Magistrate Judge to Raise You
1) Build an ESI protocol early (and treat it like a living document)
If you wait until discovery blows up to negotiate an ESI protocol, you’re basically planning to negotiate it under the threat of
court intervention. Instead, build the basics early:
- Custodians and data sources
- Date ranges and file types
- Search approach (keywords, TAR, analytics, or hybrid)
- Sampling and validation steps
- Production format and metadata fields
- Privilege handling and error correction procedures
2) If you use TAR, document your workflow like you’ll have to explain it (because you might)
A defensible TAR process is usually less about the brand of the tool and more about how you ran it. Keep records of:
- Training decisions and reviewer qualifications
- Sampling methods and results
- Iterations and adjustments
- Quality control checks
- How disagreements were resolved
3) Consider cost sharing or cost-aware proposals before someone else proposes them for you
If costs are driving disputes, say sothen propose something practical: phased discovery, sampling first, shared vendor costs,
or a targeted TAR workflow. Judges don’t love solving invoice-driven drama, but they will do it if forced.
4) Treat metadata as part of usability, not an optional accessory
Native plus metadata often makes ESI more searchable and verifiable. If you want the other side to trust your production,
produce in a format that preserves context and doesn’t create unnecessary obstacles.
5) Plan for mistakes: build in correction, clawback, and cleanup steps
The order’s “return or destroy irrelevant documents” requirement is a reminder that discovery is iterative. The best protocols
anticipate what happens when:
- search terms are refined,
- responsiveness standards evolve,
- the first pass reveals new issues, and
- nonresponsive material sneaks into production.
Common TAR Myths (That Keep Showing Up Like Spam Emails)
Myth: TAR is a shortcut that guarantees perfection
Reality: TAR is a method to improve efficiency and consistency, not a magic wand. It still needs human judgment, validation,
and quality control.
Myth: Keywords are “safer” because they’re traditional
Reality: keyword searching can be under-inclusive (missing relevant documents) or over-inclusive (pulling in mountains of junk).
TAR can be more measurable when parties validate and document the process.
Myth: If we disagree on everything, TAR can’t work
Reality: parties don’t need to like each other. They need a process that is transparent enough to be defensible and structured
enough to be testable.
FAQ: Quick Answers for Busy Humans
Can a court force parties to use TAR?
Courts have broad discretion to manage discovery methods and proportionality. Sometimes TAR is ordered, sometimes it’s endorsed,
and sometimes parties adopt it by agreement. The takeaway is that courts increasingly view TAR as a mainstream, reasonable option,
especially in large ESI disputes.
What does “statistically valid random sample” usually mean in discovery?
It generally means the sample is chosen randomly and sized in a way that supports reliable conclusions about the larger set.
The details depend on the dataset and the questions being tested, and teams often involve eDiscovery vendors or statisticians
to ensure the approach is defensible.
Is “clawback of irrelevant documents” normal?
It’s less common than privilege clawback, but it’s a practical way to handle inevitable over-production during iterative refinement.
This case highlights that courts may build “clean-up” steps into ESI protocols when dysfunction makes precision unrealistic.
Experiences From the Trenches: 500-ish Words of “Yep, This Happens”
If you’ve spent any time around eDiscovery, you’ve probably seen a version of this storymaybe without the poetic “epic”
language, but with the same ingredients: an overstuffed dataset, two sides interpreting “reasonable” like it’s a choose-your-own-adventure,
and a meet-and-confer that ends with everyone swearing they’re the only adult in the room.
Here are some common, real-world experiences that map directly to why courts keep endorsing TAR when discovery turns dysfunctional:
1) The “keyword tug-of-war”
One team proposes broad search terms (“anything that breathes might be relevant”), while the other proposes narrow terms
(“if the document doesn’t literally say ‘trade secret’ in 72-point font, it doesn’t count”). The result is two wildly different hit counts.
TAR helps because it shifts the conversation from arguing about words to testing results.
You start with a random sample, label what’s actually responsive, then let the system surface similar materialwhile sampling
to validate performance. It’s less “trust me” and more “here’s the data.”
2) The “vendor Olympics”
Sometimes the lawyers are ready to cooperate, but the vendors (or internal tech teams) aren’t aligned. One side wants TAR,
another side wants linear review, and a third person just wants everyone to stop touching the database. When a court orders
cost sharing, the tone often changes overnight. Suddenly, everyone is interested in a process that is efficient, measurable,
and not billed like a luxury vacation.
3) The “metadata is optional” misunderstanding
Producing without metadata can feel like handing someone a book after ripping out the title page, chapter headings, and page numbers.
In practice, missing metadata triggers follow-up disputes, authenticity questions, and requests to re-produce. A court pushing
native-plus-metadata can prevent a second wave of fighting that costs more than doing it right the first time.
4) The “we can’t agree on anything” reality
Dysfunctional discovery often isn’t about one issueit’s seven issues stacked in a trench coat. Who reviews? How many reviewers?
What’s the sample size? Who pays? What happens when irrelevant docs slip through? The Everlast Roofing order reads like a
checklist of the fights that show up in real cases. The lesson is not “be perfect.” It’s “build a protocol that anticipates friction.”
5) The “irrelevant docs got produced” panic
In iterative workflows, some irrelevant documents can slip into productionespecially while refining parameters.
Teams often experience a moment of panic: “Did we just ruin the case?” Courts recognize that this can happen and may require
procedures to return or destroy irrelevant information. The healthier mindset is: build guardrails, document what happened,
correct quickly, and keep the process moving.
Bottom line: TAR doesn’t fix bad behavior, but it gives the parties a structured, testable path forward. And when discovery is
melting down, judges tend to favor methods that replace endless argument with measurable progress.
Conclusion: TAR Wins When Cooperation Loses
The “epic of dysfunctional discovery” phrase is memorable, but the decision’s real value is practical: the court endorsed a
defensible pathstatistically valid sampling plus TARpaired with cost sharing, metadata-forward production, mutual inspection,
and a cleanup mechanism for irrelevant documents. It’s a reminder that discovery is not a vibes-based activity.
If you want the shortest possible summary, it’s this: courts will let you fight about discoveryuntil they won’t.
When the judge has to step in, the solution is often the same: cooperate, measure, validate, and move on. And if you refuse,
don’t be surprised if the court writes your protocol for you… with the kind of tone that makes partners suddenly very interested
in scheduling an emergency “alignment meeting.”