Virtual musculoskeletal arm and robotic arm driven by a biomimetic model of sensorimotor cortex with reinforcement learning

Salvador Dura-Bernal, George L. Chadderdon, Samuel A. Neymotin, Xianlian Zhou, Andrzej Przekwas, Joseph T. Francis, William W. Lytton

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Neocortical mechanisms of learning sensorimotor control involve a complex series of interactions at multiple levels, from synaptic mechanisms to network connectomics. We developed a model of sensory and motor cortex consisting of several hundred spiking model-neurons. A biomimetic model (BMM) was trained using spike-timing dependent reinforcement learning to drive a simple kinematic two-joint virtual arm in a motor task requiring convergence on a single target. After learning, networks demonstrated retention of behaviorally-relevant memories by utilizing proprioceptive information to perform reach-to-target from multiple starting positions. We utilized the output of this model to drive mirroring motion of a robotic arm. In order to improve the biological realism of the motor control system, we replaced the simple virtual arm model with a realistic virtual musculoskeletal arm which was interposed between the BMM and the robot arm. The virtual musculoskeletal arm received input from the BMM signaling neural excitation for each muscle. It then fed back realistic proprioceptive information, including muscle fiber length and joint angles, which were employed in the reinforcement learning process. The limb position information was also used to control the robotic arm, leading to more realistic movements. This work explores the use of reinforcement learning in a spiking model of sensorimotor cortex and how this is affected by the bidirectional interaction with the kinematics and dynamic constraints of a realistic musculoskeletal arm model. It also paves the way towards a full closed-loop biomimetic brain-effector system that can be incorporated in a neural decoder for prosthetic control, and used for developing biomimetic learning algorithms for controlling real-time devices. Additionally, utilizing biomimetic neuronal modeling in brain-machine interfaces offers the possibility for finer control of prosthetics, and the ability to better understand the brain.

Original languageEnglish (US)
Title of host publication2013 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2013
PublisherIEEE Computer Society
ISBN (Print)9781479930074
DOIs
StatePublished - Jan 1 2013
Externally publishedYes
Event2013 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2013 - Brooklyn, NY, United States
Duration: Dec 7 2013Dec 7 2013

Publication series

Name2013 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2013

Conference

Conference2013 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2013
CountryUnited States
CityBrooklyn, NY
Period12/7/1312/7/13

All Science Journal Classification (ASJC) codes

  • Biomedical Engineering

Fingerprint Dive into the research topics of 'Virtual musculoskeletal arm and robotic arm driven by a biomimetic model of sensorimotor cortex with reinforcement learning'. Together they form a unique fingerprint.

Cite this