CIF: Medium: Assessment and Modeling of Temporal Variation in Perceived Audio and Video Quality Using Direct Brainwave Measurement

The goal of this project is to better model the human perception of quality variations in audio, video, and audiovisual signals. A fundamental challenge in accomplishing this goal, however, is to develop a human subjective testing methodology that can assess such the perceived quality variations with sufficient accuracy in time. Rather than conducting human trials that simply ask subjects to rate the signal quality at given moments in time, in this research the electrical patterns of each subject's brain responses to multimedia signals of varying quality are captured using a high resolution electroencephalograph (EEG). By analyzing these EEG signals, it becomes possible to detect a change in the perceived quality of the signal before the human observer/listener even becomes consciously aware that the quality has changed.

A major difficulty, however, is that the EEG waveforms captured during these trials contain large amounts of noise, and it is therefore necessary to sift through a large set of data to identify the components of the collected EEG waveforms that correspond to changes in perceived quality. To accomplish this task, both deterministic time-space-frequency analysis techniques will be applied as well as stochastic techniques based on information spectra. To create computer-based models of perceived quality, AR/ARMA (autoregressive/autoregressive moving average) modeling techniques will be considered and support vector machines along with related kernel-based classifiers will be designed to output class indexes corresponding to perceived quality. Beyond its potential for improving audiovisual transmission systems, the broader impacts of this research include the possibility that it will open up new and more efficient avenues for transferring information from computers to human beings.