Vision Transformer-based driver fatigue detection in real-time embedded systems

Abstract

Driver fatigue is a major contributor to traffic accidents, yet reliable early warning remains challenging under real-world and embedded constraints. This study presents a camera-only driver-fatigue detection pipeline that couples lightweight face tracking (MediaPipe) with a single Vision Transformer (ViT) classifier ( $$\approx$$ 86M parameters) to infer alert vs. drowsy states from facial cues. The model is trained on a multi-source dataset of $$\sim$$ 4,000 images aggregated from seven public fatigue datasets and evaluated on a held-out test split, achieving 95.0% accuracy, 93.8% F1-score, and ROC-AUC of 0.986. On Raspberry Pi 5, end-to-end inference runs at 55.6 ms/frame (18.0 FPS). Compared with a detection-driven YOLO baseline using the same labeling protocol and test split, the proposed ViT improves accuracy (95.0% vs. 93.2%) and ROC-AUC (0.986 vs. 0.979) while trading throughput (18.0 vs. 28.0 FPS), clarifying the accuracy–latency balance for deployment. Real-world validation with 12 participants (22–47 years) across day and night sessions ( $$\sim$$ 9 hours total) shows high agreement with manual annotations (92.8%), supporting operational feasibility. By combining a single-backbone ViT inference pipeline with explicit embedded benchmarking, matched baseline comparison, and in-vehicle validation, the proposed system provides a deployment-oriented evaluation package for camera-only fatigue monitoring under embedded ITS constraints.