TY - JOUR
T1 - Spiking Neural Networks - Part II
T2 - Detecting Spatio-Temporal Patterns
AU - Skatchkovsky, Nicolas
AU - Jang, Hyeryung
AU - Simeone, Osvaldo
N1 - Funding Information:
December 27, 2020. Date of publication January 8, 2021; date of current version June 10, 2021. This work has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation Programme (Grant Agreement No. 725731). This work was In this section, we first review the details of the deterministic done when H. Jang was with King’s College London. The associate editor discrete-time Spike Response Model (SRM) introduced in D.Ciuonzo. (NicolasSkatchkovsky andHyeryungJangcontributed equallycoordinatingthe reviewofthisletter andapprovingit for publicationwas Part I, and then describe a probabilistic extension of the SRM to this work.) (Corresponding author: Nicolas Skatchkovsky.) known as Generalized Linear Model (GLM). Throughout this Nicolas Skatchkovsky and Osvaldo Simeone are with Department of letter, we consider a general SNN architecture described by nicolas.skatchkovsky@kcl.ac.uk;osvaldo.simeone@kcl.ac.uk).Engineering,King’s CollegeLondon,LondonWC2R2LS, U.K. (e-mail: a directed, possibly cyclic, graph with spiking neurons as Hyeryung Jang was with the Department of Engineering, King’s Col-vertices and synapses as directed edges. As seen in Fig. 1, lege London, London WC2R 2LS, U.K. She is now with the Department each neuron i receives synaptic connections from a subset Pi (e-mail:hyeryung.jang@dgu.ac.kr).of ArtificialIntelligence,DonggukUniversity, Seoul 04620, South Korea of other spiking neurons. With a slight abuse of notations, Digital Object Identifier 10.1109/LCOMM.2021.3050242 we take the set Pi to include also exogeneous inputs. 1558-2558 © 2021 I EEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher Copyright:
© 1997-2012 IEEE.
PY - 2021/6
Y1 - 2021/6
N2 - Inspired by the operation of biological brains, Spiking Neural Networks (SNNs) have the unique ability to detect information encoded in spatio-temporal patterns of spiking signals. Examples of data types requiring spatio-temporal processing include logs of time stamps, e.g., of tweets, and outputs of neural prostheses and neuromorphic sensors. In this letter, the second of a series of three review papers on SNNs, we first review models and training algorithms for the dominant approach that considers SNNs as a Recurrent Neural Network (RNN) and adapt learning rules based on backpropagation through time to the requirements of SNNs. In order to tackle the non-differentiability of the spiking mechanism, state-of-the-art solutions use surrogate gradients that approximate the threshold activation function with a differentiable function. Then, we describe an alternative approach that relies on probabilistic models for spiking neurons, allowing the derivation of local learning rules via stochastic estimates of the gradient. Finally, experiments are provided for neuromorphic data sets, yielding insights on accuracy and convergence under different SNN models.
AB - Inspired by the operation of biological brains, Spiking Neural Networks (SNNs) have the unique ability to detect information encoded in spatio-temporal patterns of spiking signals. Examples of data types requiring spatio-temporal processing include logs of time stamps, e.g., of tweets, and outputs of neural prostheses and neuromorphic sensors. In this letter, the second of a series of three review papers on SNNs, we first review models and training algorithms for the dominant approach that considers SNNs as a Recurrent Neural Network (RNN) and adapt learning rules based on backpropagation through time to the requirements of SNNs. In order to tackle the non-differentiability of the spiking mechanism, state-of-the-art solutions use surrogate gradients that approximate the threshold activation function with a differentiable function. Then, we describe an alternative approach that relies on probabilistic models for spiking neurons, allowing the derivation of local learning rules via stochastic estimates of the gradient. Finally, experiments are provided for neuromorphic data sets, yielding insights on accuracy and convergence under different SNN models.
KW - Neuromorphic computing
KW - spiking neural networks (SNNs)
UR - http://www.scopus.com/inward/record.url?scp=85099569279&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85099569279&partnerID=8YFLogxK
U2 - 10.1109/LCOMM.2021.3050242
DO - 10.1109/LCOMM.2021.3050242
M3 - Article
AN - SCOPUS:85099569279
SN - 1089-7798
VL - 25
SP - 1741
EP - 1745
JO - IEEE Communications Letters
JF - IEEE Communications Letters
IS - 6
M1 - 9317741
ER -