Synthetic X‑ray‑driven tracking and control of miniature medical devices

Abstract

The clinical translation of miniature medical devices (MMDs) for minimally invasive surgery promises transformative advances in biomedical engineering, offering enhanced precision, reduced patient trauma and faster recovery times. However, their effective deployment in complex anatomies under real-time X-ray guidance—a widely used surgical imaging modality—presents challenges such as low imaging quality and difficulties of spatial MMD control. Manual identification and operation are labour intensive and error prone. Meanwhile, deep learning-based automation is limited by the scarcity of annotated X-ray datasets of MMDs owing to costly data collection, laborious annotation and privacy constraints. Here we introduce MicroSyn-X, a framework for training computer vision models to enable robotic teleoperation of MMDs using synthesized high-fidelity, pixel-accurate, auto-labelled and domain-randomized X-ray images, eliminating manual data curation. Integrating MicroSyn-X into a teleoperated robotic system enables real-time localization and navigation of magnetic soft and magnetic liquid MMDs within both ex vivo and dynamic in vivo environments, demonstrating robustness under challenging imaging conditions of low contrast, high noise and occlusion. With these promises, we open source the X-ray MMD dataset to enable benchmarking. Addressing data scarcity and enabling real-time robotic navigation, this work advances MMD-assisted minimally invasive surgery towards next-generation precision interventions. © The Author(s) 2026.