Adaptive micro-locomotion in a dynamically changing environment via context detection

Substantial efforts have exploited reinforcement learning (RL) in the development of micro-robotic locomotion. These RL-powered micro-robots are capable of learning a locomotory policy based on their experience interacting with the surroundings, without requiring prior knowledge on the physics of locomotion in that environment. However, in their applications, micro-robots often encounter changes in the environment and need to adapt their locomotory gaits like living organisms in order to achieve robust locomotion performance. In standard RL methods, such a non-stationary environment can cause the micro-robots to continuously relearn the policy from scratch, degrading their locomotion performance. In this work, we explore a first use of a recently developed context detection method combined with deep RL to facilitate micro-robotic locomotion in a dynamically changing environment. As a proof-of-principle, we consider a simple micro-robot immersed in non-stationary environments switching between a viscous fluid environment and a dry frictional environment. We show that the RL with context detection approach enables the micro-robot to effectively detect changes in the environment and deploy specialized locomotory gaits for different environments accordingly to achieve significantly improved locomotion. Our results suggest the integration of deep RL with context detection as a potential tool for robust micro-robotic locomotion across different environments.