Curriculum based reinforcement learning for 3D control of magnetic microrobot swarms

Abstract

Microrobotic swarms (MS) have shown great potential in a variety of biomedical applications, including targeted drug delivery and minimally invasive surgery. However, controlling MS in 3D environments remains a significant challenge. In this study, we propose a curriculum-based reinforcement learning (RL) approach for the autonomous navigation and control of MS in a 3D space, utilizing a magnetic field gradient for actuation and a field-free point to manage swarm formation. The RL agent learns to control the swarm's position and minimize dispersion, progressively moving from 2D to 3D environments, and finally handling up to 8 microrobots. We further integrate the $A^{*}$ algorithm with the Artificial Potential Field (APF) method to manage path planning in environments with static and dynamic obstacles. The results demonstrate the effectiveness of the proposed approach, showing that the MS can autonomously navigate to a target with minimal dispersion and avoid obstacles in real-time. Comparison with human control highlights the advantages of the RL-based strategy in terms of efficiency and consistency. This work lays the foundation for future advancements in autonomous microrobotic swarm systems, offering a promising solution for complex, real-world applications.