Lawyers don't just ask what a case decided. They ask whether it is still good law. A ruling that was strong ten years ago may have been narrowed, questioned, or overturned since. Checking this by hand means reading every later case that cites it, and for a well-known ruling that can mean thousands of decisions.
AI can help with that check, but only if it learns from carefully labeled examples. Good case law citation annotation means recording how one case treats another, the same way every time. This post explains what the work covers, why it is hard, and what legal AI training data needs to get right.
Annotation for case law usually has four parts:
Models trained on these labels can warn a researcher that a case has been overruled, rank authorities by strength, and surface the cases that matter.
A treatment label describes the relationship between two cases. Common labels include:
The label set differs between legal systems and publishers, so the guide should follow the one the project needs, with a definition and example for each. Every label should also mark the supporting passage. This lets reviewers check it quickly, helps the model learn which words signal each treatment, and lets a lawyer read the court’s own words. Where no clear passage exists, use “cited neutrally” or “unclear,” not a guess based on the outcome.
Judges do not write in tidy labels.
Annotators need rules for each of these. Without them, two people label the same passage differently, and the model learns the disagreement.
Citation work is about identification. Labelers find the reference and link it to the right case, and errors come from short names, typing variants, and cases that share a name. Treatment work is about judgment. Labelers read the surrounding reasoning and decide how the court used the earlier case, and errors come from soft wording.
A single guide for both would blur these differences. Each needs its own definitions, examples, and reviewers.
Treatment depends on who is speaking. A higher court can overrule a lower one, and a court generally cannot overrule itself without special steps. A lower court that disagrees may say so, but the earlier ruling still stands.
Each record should carry the court level, date, and jurisdiction of both cases. “Overruled” should only be assigned when the later court had the authority; otherwise, record it as criticized or not followed.
Many decisions treat an earlier case in parts, accepting one rule and rejecting another. The guide should say whether to assign one overall label or several labels tied to specific points. Point-level labels are more useful but slower.
Law changes, so labels must carry dates. Most citations are neutral, and strong signals such as overruling are rare, so count examples for each label and collect more of the thin ones early. Keep decisions from the same dispute on one side of the train and test split, or scores will look better than they are.
Some passages cannot support a label. Give annotators an “unclear” option and send doubtful cases to a legal reviewer. A wrong “overruled” label is serious, so the rules should say how doubt is handled. Our guide to eDiscovery document review labeling covers a related need to separate clear calls from judgment calls in legal text.
People with legal training know how courts signal disagreement, how precedent works across court levels, and why “distinguished” is not the same as “criticized.” They can tell a passing mention from a real application of a rule.
Specialists write the treatment definitions, set the court-level rules, and review hard cases. This reflects Globik AI’s approach of using verified domain experts for work that depends on judgment, where a generalist would be guessing.
Before starting a case law project, settle these points:
A case is only as strong as the later decisions that treat it, and the label must match what the court actually did. Strong case law citation annotation uses clear treatment definitions, court-level rules, marked evidence, and reviewers with legal training. With that foundation, models can help researchers see which authorities still stand.
It is the labeling of references between court decisions so AI can find each citation and record how the later case treats the earlier one. Annotators follow written legal definitions.
The later court says the earlier case does not apply, usually because the facts or legal issue differ. It does not reject the earlier rule, which is how it differs from overruling.
Only a court with enough authority can overrule an earlier decision. Labels must record court level and jurisdiction so a lower court’s disagreement is not marked as overruling.
It lets reviewers check the label quickly and helps the model learn which wording signals each treatment. Without it, labels are hard to trust.
Specialists understand precedent, court levels, and how judges signal disagreement. They write clear definitions and review doubtful passages.
