NormAD - Normalized Approximate Descent based supervised learning rule for spiking neurons

Navin Anwani, Bipin Rajendran

Research output: Chapter in Book/Report/Conference proceedingConference contribution

23 Scopus citations

Abstract

NormAD is a novel supervised learning algorithm to train spiking neurons to produce a desired spike train in response to a given input. It is shown that NormAD provides faster convergence than state-of-the-art supervised learning algorithms for spiking neurons, often the gain in the rate of convergence being more than a factor of 10. The algorithm leverages the fact that a leaky integrate-and-fire neuron can be described as a non-linear spatio-temporal filter, allowing us to treat supervised learning as a mathematically tractable optimization problem with a cost function in terms of the membrane potential rather than the spike arrival time. A variant of stochastic gradient descent along with normalization has been used to derive the synaptic weight update rule. NormAD uses leaky integration of the input to determine the synaptic weight change. Since leaky integration is fundamental to all integrate-and-fire models of spiking neurons, we claim universal applicability of the learning rule to other models such as adaptive exponential integrate-and-fire model of neurons by demonstrating equally good performance in training with our algorithm.

Original languageEnglish (US)
Title of host publication2015 International Joint Conference on Neural Networks, IJCNN 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781479919604, 9781479919604, 9781479919604, 9781479919604
DOIs
StatePublished - Sep 28 2015
Externally publishedYes
EventInternational Joint Conference on Neural Networks, IJCNN 2015 - Killarney, Ireland
Duration: Jul 12 2015Jul 17 2015

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2015-September

Other

OtherInternational Joint Conference on Neural Networks, IJCNN 2015
Country/TerritoryIreland
CityKillarney
Period7/12/157/17/15

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Keywords

  • LIF
  • NormAD
  • SNN
  • leaky integrate-and-fire
  • normalized approximate descent
  • spiking neuron
  • supervised learning

Fingerprint

Dive into the research topics of 'NormAD - Normalized Approximate Descent based supervised learning rule for spiking neurons'. Together they form a unique fingerprint.

Cite this