Most conversations about vehicle AI look outward, at the road, the lanes, and the traffic. A second set of systems looks inward. A small camera in the cabin watches whether the driver is paying attention, whether seatbelts are fastened, and who is sitting where, including a child in the back.
Those systems learn from labeled images and video of real cabins, and the labels are demanding. Driver monitoring annotation is the work of turning cabin footage into reliable examples of attention, posture, and occupancy. This post explains what the labels need to capture, why they are hard to get right, and why careful definitions matter more than speed.
Cabin cameras usually work in near-infrared light, so the picture is grayscale and looks the same by day and by night.
A common first layer marks the face and head: where the eyes and mouth are, which way the head points, and whether the eyes are open, closed, or partly closed.
Many things get in the way. Sunglasses hide the eyes. A cap shades them. A hand covers the mouth. A mask, a scarf, or a head covering changes the visible shape of the face. Annotators need a rule for each case and a "cannot see" label, because a model should learn to say it cannot tell, not to guess.
Eye state needs a defined scale. Is "half closed" a separate class, or a percentage? Without a rule, two annotators will place the same frame in different classes.
Attention is the heart of the system, and no camera sees it directly. What annotators can label is where the driver appears to be looking.
The usual approach divides the view into zones: the road ahead, the mirrors, the instrument cluster, the center screen, the side windows, and the lap or phone area. Annotators assign a zone to each frame or segment.
The difficulty is that head direction and eye direction are not the same. Cameras often cannot resolve gaze exactly. Good projects use coarse zones, define how to treat head-only evidence, and include an "uncertain" option. They also keep calibration data, such as sessions where the driver looked at known targets, so that labels can be checked against something real.
Drowsiness and distraction sound like single things. In footage, they are collections of cues.
For drowsiness, cues may include long eye closures, head nods, and yawning. For distraction, they may include phone use, eating, drinking, reaching for objects, or turning to a passenger. Each cue needs a written definition, and each needs a duration rule. A normal blink is not drowsiness, and a quick glance at the mirror is not distraction.
Labels should describe the visible behavior and not claim to measure a person's inner state.
Staged data raises its own issue. Actors can pretend to be sleepy, but it never looks quite like the real thing. Datasets should record whether a clip was acted or naturally recorded, so that the difference stays visible to the people training the model.
Belt detection sounds simple until you look at the footage. A belt can be worn correctly, pulled across the body, tucked under an arm, placed behind the back, or left unbuckled but laid across the lap. A dark belt on dark clothing nearly disappears in infrared. Coats, bags, and arms can hide it.
Seatbelt detection labeling needs a defined list of belt states with example frames for each, and a "cannot see" label for the cases where clothing or posture hides the belt.
Occupancy is the other half. Annotators label who or what is in each seat: an adult, a child, an infant in a child seat, a pet, or an object such as a bag. Child seats come in forward-facing and rear-facing forms, which look very different. A rule is needed for a child half hidden by a headrest, and for a coat on a seat that resembles a small person.
Cameras sit in different places: on the mirror, the steering column, the pillar, or the roof. Each position gives a different angle and a different view of the cabin. Seat positions, headrests, and dashboards vary between models.
A label set should record the camera position and vehicle type for every clip.
A driver monitoring model has to work for everyone who drives. Faces, skin tones, ages, glasses, facial hair, and head coverings all affect how a face appears to a camera.
The same guidelines should apply to every face, and the team should check whether certain groups get more "cannot see" labels or more disagreement than others. A pattern there points to a gap in the data.
In-cabin footage shows a person's face in a private space. It is personal data and should be treated that way.
Projects should settle consent, how long raw footage is kept, who can view it, and whether faces are masked where the task allows. Labels should refer to the person's behavior, not their identity.
People with experience in human factors or vehicle safety read cabin footage differently. They understand how attention and fatigue show up in behavior, know which seatbelt states matter, and can tell a real concern from a harmless movement. They also help write the rules that keep a large team consistent: the zone map, the eye state scale, the duration rules, and the handling of unclear frames.
This matches Globik's approach of using verified domain experts for work that depends on judgment. For automotive AI training data, that knowledge is what gives a monitoring system labels it can be tested against.
Before an in-cabin labeling project starts, a team can settle five things:
In-cabin AI works when labels describe visible behavior, admit what cannot be seen, separate acted from natural footage, and respect the people in the car. If you are building a monitoring system, you can talk to our team about your in-cabin data. Strong in-cabin monitoring data depends on people who understand how drivers behave.
It is the labeling of cabin footage so AI can recognize attention, behavior, seatbelt use, and who is in each seat.
Cameras cannot measure gaze exactly, so coarse zones with an "uncertain" option give more reliable labels.
Staged drowsiness looks different from the real thing, so the difference must stay visible.
Belts can be worn wrongly or blend into dark clothing in infrared, so states and a "cannot see" label are needed.
It shows faces in a private space, so consent, access, and retention must be settled first.
