Spiking neural networks for handwritten digit recognition—Supervised learning and network optimization

Shruti R. Kulkarni, Bipin Rajendran

Research output: Contribution to journalArticlepeer-review

124 Scopus citations

Abstract

We demonstrate supervised learning in Spiking Neural Networks (SNNs) for the problem of handwritten digit recognition using the spike triggered Normalized Approximate Descent (NormAD) algorithm. Our network that employs neurons operating at sparse biological spike rates below 300Hz achieves a classification accuracy of 98.17% on the MNIST test database with four times fewer parameters compared to the state-of-the-art. We present several insights from extensive numerical experiments regarding optimization of learning parameters and network configuration to improve its accuracy. We also describe a number of strategies to optimize the SNN for implementation in memory and energy constrained hardware, including approximations in computing the neuronal dynamics and reduced precision in storing the synaptic weights. Experiments reveal that even with 3-bit synaptic weights, the classification accuracy of the designed SNN does not degrade beyond 1% as compared to the floating-point baseline. Further, the proposed SNN, which is trained based on the precise spike timing information outperforms an equivalent non-spiking artificial neural network (ANN) trained using back propagation, especially at low bit precision. Thus, our study shows the potential for realizing efficient neuromorphic systems that use spike based information encoding and learning for real-world applications.

Original languageEnglish (US)
Pages (from-to)118-127
Number of pages10
JournalNeural Networks
Volume103
DOIs
StatePublished - Jul 2018

All Science Journal Classification (ASJC) codes

  • Cognitive Neuroscience
  • Artificial Intelligence

Keywords

  • Approximate computing
  • Neural networks
  • Neuromorphic computing
  • Pattern recognition
  • Spiking neurons
  • Supervised learning

Fingerprint

Dive into the research topics of 'Spiking neural networks for handwritten digit recognition—Supervised learning and network optimization'. Together they form a unique fingerprint.

Cite this