A single CT scan can contain hundreds of images. An MRI study can contain several sets of them, each showing the body in a different way. Radiologists read this volume of material every day, and AI is increasingly used to help: measuring findings, highlighting areas of concern, and saving time on routine checks.
The AI learns from examples that were outlined and measured by people who understand the images. Radiology image annotation is the work of producing those examples. It is more than drawing around an organ. This post explains what the labels need to capture, why scans are harder to label than ordinary pictures, and why precise protocols matter. In every case, AI supports clinicians, and diagnoses remain theirs to make.
CT and MRI scans are not single photographs. They are stacks of thin slices, taken one after another through the body.
CT and MRI annotation therefore works in three dimensions. A finding that appears on one slice continues on the next, growing, shrinking, or changing shape. The label has to follow it through the whole stack, not sit on a single image.
Annotators who treat each slice as a separate picture produce outlines that jump from one slice to the next. The model then learns a shape that does not exist in the body.
In medical image segmentation, an annotator marks every pixel that belongs to a structure or finding. For a volume, that means outlining it on many slices.
Most tools help by interpolating between slices the annotator has outlined. That saves time, but it also spreads errors. Interpolation can cut across a real edge or miss a narrow part. Good protocols require annotators to check the result and correct it.
Scans can be viewed in three directions: from above, from the front, and from the side. Reviewing in all three catches errors that a single view hides.
A CT image does not show a fixed picture. The viewer chooses a window, a range of values that is stretched across the grayscale. One window shows lung detail. Another shows soft tissue. Another shows bone. The same slice looks very different under each.
An annotator who labels a finding in the wrong window may miss part of it or include tissue that does not belong. Protocols should specify which windows to use for which structures, and projects should record the settings used for each label.
MRI adds a further layer. Different sequences show different properties of tissue, and a finding may be clear in one and faint in another. Annotators need to know which sequences to use and how to compare them.
Medical images follow display conventions. In one common convention, the patient's right appears on the left side of the screen, as if the viewer were facing the patient. Other systems differ.
Orientation is also stored in the file, and software can flip or rotate images. An annotator who assumes the wrong side can label the opposite lung or kidney. Laterality errors are among the most serious mistakes in this field.
Projects should state the display convention, check orientation against the file's information, and review a sample of labels for side errors before scaling up.
Edges in scans are not sharp. A finding fades into the tissue around it. Some findings have layers: a central part, a surrounding zone, and swelling around both.
The protocol has to say where to draw the boundary and how to handle sub-regions. Without a rule, two annotators outline the same finding at different sizes. The model learns an average that fits neither, and any measurement taken from it is unreliable.
Many imaging tasks depend on measurements: the longest diameter of a finding, its volume, or the distance between two structures. These measurements are labels, and they need rules.
Where is the diameter measured? On which slice? In which direction? A different choice gives a different number. Protocols should define the method and ask annotators to record the slice where each measurement was taken, so that results can be checked later.
Scans contain marks that are not findings. Patient movement blurs an image. Metal implants cause streaks. Noise and reconstruction effects create patterns that resemble real structures.
There are also normal variants: harmless features that differ between people but look unusual to the untrained eye. A model trained on these as findings will raise false alarms.
Annotators need examples of both, and a "cannot assess" label for slices where artifacts hide the area. A forced label is worse than an honest gap.
Many datasets use radiology reports to create labels, since reports are written by specialists and already exist. They have limits. A report may mention a finding without saying which slice it is on. It may describe the whole study in words that do not point to a location.
Report-derived labels should be treated as a starting point. Where a task needs the location of a finding, a specialist should confirm it on the images.
Radiologists, radiographers, and medical imaging specialists see scans in ways that generalists do not. They know the anatomy, understand windowing and sequences, recognize artifacts and normal variants, and know which side of the body the image shows. They write the protocols that keep a large team consistent: which structures to label, how to draw edges, how to measure, and when to say "cannot assess."
This matches Globik's approach of using verified domain experts for work that depends on judgment. For healthcare imaging AI training data, that expertise is what makes labels precise enough to build on. You can read more in our healthcare AI training data services.
Before an imaging annotation project starts, a team can settle five things:
Radiology AI works when labels follow findings through the whole volume, respect display settings and orientation, define edges and measurements clearly, and admit what cannot be assessed. Strong radiology image annotation depends on people who read scans every day.
It is the outlining and measuring of findings in medical scans so AI can learn to recognize them.
Scans are stacks of slices, so a finding must be followed through the whole volume.
Different windows and sequences show different tissues, so a label made in the wrong one can miss part of a finding.
Display conventions and file orientation can differ, and labeling the wrong side is a major mistake.
They are a starting point, since reports often lack locations and need specialist confirmation on the images.
