TY - JOUR
T1 - Mechanical rotation at low Reynolds number via reinforcement learning
AU - Liu, Yuexin
AU - Zou, Zonghao
AU - Tsang, Alan Chen Hou
AU - Pak, On Shun
AU - Young, Y. N.
N1 - Funding Information:
Funding support by the National Science Foundation (Grant No. EFMA-1830958 to O.S.P. and Grant Nos. 1614863 and 1951600 to Y.-N.Y.) is gratefully acknowledged. Y.-N.Y. acknowledges support from Flatiron Institute, part of Simons Foundation. Z.Z. and O.S.P. also acknowledge the computational resources from the WAVE computing facility (enabled by the E. L. Wiegand Foundation) at Santa Clara University.
Publisher Copyright:
© 2021 Author(s).
PY - 2021/6/1
Y1 - 2021/6/1
N2 - There is growing interest in the development of artificial microscopic machines that can perform complex maneuvers like swimming microorganisms for potential biomedical applications. At the microscopic scales, the dominance of viscous over inertial forces imposes stringent constraints on locomotion. In the absence of inertia, Purcell first proposed an elegant way to generate net translation using kinematically irreversible motions [E. M. Purcell, “Life at low Reynolds number,” Am. J. Phys. 45, 3-11 (1977)]. In addition to net translation, a more recent prototype known as Purcell's “rotator” has been proposed in Dreyfus et al. [“Purcell's “rotator”: Mechanical rotation at low Reynolds number,” Eur. Phys. J. B 47, 161-164 (2005)] as a mechanical implementation of net rotation at low Reynolds numbers. These ingenious designs rely on knowledge of the surrounding environment and the physics of locomotion within the environment, which may be incomplete or unclear in more complex scenarios. More recently, reinforcement learning has been used as an alternative approach to enable a machine to learn effective locomotory gaits for net translation based on its interaction with the surroundings. In this work, we demonstrate the use of reinforcement learning to generate net mechanical rotation at low Reynolds numbers without requiring prior knowledge of locomotion. For a three-sphere configuration, the reinforcement learning recovers the strategy proposed by Dreyfus et al. As the number of spheres increases, multiple effective rotational strategies emerge from the learning process. However, given sufficiently long learning processes, all machines considered in this work converge to a single type of rotational policies that consist of traveling waves of actuation, suggesting its optimality of the strategy in generating net rotation at low Reynolds numbers.
AB - There is growing interest in the development of artificial microscopic machines that can perform complex maneuvers like swimming microorganisms for potential biomedical applications. At the microscopic scales, the dominance of viscous over inertial forces imposes stringent constraints on locomotion. In the absence of inertia, Purcell first proposed an elegant way to generate net translation using kinematically irreversible motions [E. M. Purcell, “Life at low Reynolds number,” Am. J. Phys. 45, 3-11 (1977)]. In addition to net translation, a more recent prototype known as Purcell's “rotator” has been proposed in Dreyfus et al. [“Purcell's “rotator”: Mechanical rotation at low Reynolds number,” Eur. Phys. J. B 47, 161-164 (2005)] as a mechanical implementation of net rotation at low Reynolds numbers. These ingenious designs rely on knowledge of the surrounding environment and the physics of locomotion within the environment, which may be incomplete or unclear in more complex scenarios. More recently, reinforcement learning has been used as an alternative approach to enable a machine to learn effective locomotory gaits for net translation based on its interaction with the surroundings. In this work, we demonstrate the use of reinforcement learning to generate net mechanical rotation at low Reynolds numbers without requiring prior knowledge of locomotion. For a three-sphere configuration, the reinforcement learning recovers the strategy proposed by Dreyfus et al. As the number of spheres increases, multiple effective rotational strategies emerge from the learning process. However, given sufficiently long learning processes, all machines considered in this work converge to a single type of rotational policies that consist of traveling waves of actuation, suggesting its optimality of the strategy in generating net rotation at low Reynolds numbers.
UR - http://www.scopus.com/inward/record.url?scp=85108086268&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85108086268&partnerID=8YFLogxK
U2 - 10.1063/5.0053563
DO - 10.1063/5.0053563
M3 - Article
AN - SCOPUS:85108086268
SN - 1070-6631
VL - 33
JO - Physics of Fluids
JF - Physics of Fluids
IS - 6
M1 - 062007
ER -