A great deal of what clinicians know about a patient is written in notes. Structured fields hold a few values, but the story sits in sentences: what the patient reported, what the examination showed, what was ruled out, and what the plan is. Software that can read those notes could help with search, summaries, and research.
Reading clinical text is harder than finding medical words in it. Clinical note annotation is the work of teaching a model what the sentences actually say. This post explains why mentions alone mislead, what extra labels notes need, and why clinical knowledge makes the difference. The labels support software that helps clinicians, and they do not replace clinical judgment.
The first layer of annotation is usually entity recognition: marking problems, medications, tests, and procedures in the text. It is necessary, and it is not enough.
Consider a note that says "no chest pain." A system that only finds mentions would record chest pain. A sentence about a relative's illness would be recorded as the patient's. A condition that the doctor wants to rule out would be recorded as present.
Each of these errors is a mismatch between a mention and a fact. The extra labels described below close that gap.
Clinical notes are full of negatives. A review of symptoms is often a list of things the patient does not have. Phrases such as "denies," "without," "negative for," and "no evidence of" all negate what follows.
Scope is the difficulty. In "no fever, cough, or chills," the negation covers three items. In "no fever but persistent cough," it covers one. Annotators must mark which mentions fall inside the negation and which do not.
Negation and assertion labeling needs worked examples of tricky constructions, because small changes in wording change the scope.
Clinicians hedge. A note may say "concern for," "cannot exclude," "may represent," or "likely." These phrases do not state a fact. They express uncertainty.
A useful label set separates several states for each mention:
Collapsing these into present or absent throws away information that clinicians chose their words carefully to convey.
Notes mention people other than the patient. A family history section lists conditions in parents and siblings. A caregiver may report symptoms. A sentence may describe a relative's illness as context.
The label for this is called the experiencer: who the finding applies to. Annotators mark whether a mention concerns the patient, a family member, or someone else. Without it, a family history of a condition appears as a diagnosis in the patient.
Time matters as much as presence. "Chest pain started this morning," "history of chest pain," and "return if chest pain recurs" are three different statements about the same symptom.
Temporality labels place each mention relative to the visit: current, past, or planned. Many notes give timing in relative terms, such as "two weeks ago" or "since childhood." Annotators should record the relationship to the visit and the duration where stated.
Exact dates raise privacy concerns. They should be handled according to the project's redaction process, which is a separate step from this labeling.
A medication mention is a bundle of facts. There is the drug, the dose, the route, and the frequency. There is also the status: started, stopped, changed, continued, or planned.
"Hold the blood pressure medication" and "continue the blood pressure medication" use the same words and mean opposite things. Annotators label each part and link them to the right drug. Allergies deserve their own handling, since an allergy to a drug is different from taking it, and a reaction is different from an allergy.
Notes follow a structure: history, examination, assessment, and plan. The same phrase can mean different things depending on where it sits. A condition in the history section is past or background. The same condition in the plan may be something to be tested.
Annotators therefore need to label sections, or at least see them, and use them in their decisions.
Notes also contain text that was copied from earlier visits or filled in from a template. Copied text may be out of date, and a template may list negatives that nobody checked. Where the project cares, annotators should mark such text, so that a model does not treat boilerplate as a fresh observation.
Because clinical language is varied, guidelines work best when they include many real examples. Each rule should say what to do with phrases like "no history of," "no known," "to rule out," and "family history unremarkable." Where annotators disagree, the group should settle the case and add it to the guide.
Over time, this builds a shared set of decisions that keeps a large team consistent.
Clinicians and people with clinical documentation experience read notes differently. They know the shorthand, understand the difference between "rule out" and "present," and recognize what belongs in each section. They can tell a hedge from a statement of fact and a template line from a real observation.
They also write the rules that keep a team consistent: the assertion categories, the experiencer labels, the status values for medications, and the handling of copied text. This matches Globik's approach of using verified domain experts for work that depends on judgment. For clinical NLP training data, that experience is what makes the labels reflect what the note actually says.
Before a clinical note project starts, a team can settle five things:
Clinical NLP works when labels capture whether something is present, whose it is, when it applies, and what the clinician meant, and not just that a medical word appeared. For another example of labeling medical data with careful protocols, see our guide on digital pathology slide annotation: labeling cells and tissue at gigapixel scale. Strong EHR NLP annotation depends on people who read clinical notes every day.
It is the labeling of clinical text so software can capture what a note says about a patient.
Phrases like "no chest pain" mention a condition the patient does not have, so a model must see the negation.
It records whether a finding applies to the patient, a family member, or someone else.
The same drug can be started, stopped, changed, or planned, and the status changes the meaning.
A phrase means different things in history and plan, and copied text may be out of date.
