M1M2: Deep-Learning-Based Real-Time Emotion Recognition from Neural Activity

Sumya Akter, Rumman Ahmed Prodhan, Tanmoy Sarkar Pias, David Eisenberg, Jorge Fresneda Fernandez

Research output: Contribution to journalArticlepeer-review

13 Scopus citations

Abstract

Emotion recognition, or the ability of computers to interpret people’s emotional states, is a very active research area with vast applications to improve people’s lives. However, most image-based emotion recognition techniques are flawed, as humans can intentionally hide their emotions by changing facial expressions. Consequently, brain signals are being used to detect human emotions with improved accuracy, but most proposed systems demonstrate poor performance as EEG signals are difficult to classify using standard machine learning and deep learning techniques. This paper proposes two convolutional neural network (CNN) models (M1: heavily parameterized CNN model and M2: lightly parameterized CNN model) coupled with elegant feature extraction methods for effective recognition. In this study, the most popular EEG benchmark dataset, the DEAP, is utilized with two of its labels, valence, and arousal, for binary classification. We use Fast Fourier Transformation to extract the frequency domain features, convolutional layers for deep features, and complementary features to represent the dataset. The M1 and M2 CNN models achieve nearly perfect accuracy of 99.89% and 99.22%, respectively, which outperform every previous state-of-the-art model. We empirically demonstrate that the M2 model requires only 2 seconds of EEG signal for 99.22% accuracy, and it can achieve over 96% accuracy with only 125 milliseconds of EEG data for valence classification. Moreover, the proposed M2 model achieves 96.8% accuracy on valence using only 10% of the training dataset, demonstrating our proposed system’s effectiveness. Documented implementation codes for every experiment are published for reproducibility.

Original languageEnglish (US)
Article number8467
JournalSensors
Volume22
Issue number21
DOIs
StatePublished - Nov 2022

All Science Journal Classification (ASJC) codes

  • Analytical Chemistry
  • Information Systems
  • Instrumentation
  • Atomic and Molecular Physics, and Optics
  • Electrical and Electronic Engineering
  • Biochemistry

Keywords

  • CNN
  • EEG
  • deep learning
  • emotion recognition
  • machine learning
  • sensor

Fingerprint

Dive into the research topics of 'M1M2: Deep-Learning-Based Real-Time Emotion Recognition from Neural Activity'. Together they form a unique fingerprint.

Cite this