Generative Modeling with Short Run Computing

In our daily lives, we constantly receive a large amount of sensory data in the form of images, texts, and speech, yet we can effortlessly make sense of the data by learning, recognizing, and understanding the patterns and meanings in the data. How this is done by the brain is still largely a mystery, and this is a central problem in machine learning and artificial intelligence, which has a vast scope of applications and is transforming our lives. One way to make sense of sensory data is to construct models to generate them, by assuming that the data are generated by some relatively simple hidden factors or causes. Such models are called generative models. To make sense of the data is to infer the hidden causes that generate the input data, and this can be accomplished by short-run computing dynamics. The main goal of this project is to develop such generative models and the associated short-run computing dynamics. The project has the potential to lead to new learning techniques that can be useful in applications such as computer vision. The PI will also train graduate students supported by this grant and further enhance the graduate and undergraduate courses taught by the PI.

Generative models unify supervised, unsupervised, and semi-supervised learning in a principled likelihood-based framework. While supervised learning has met tremendous successes in recent years, unsupervised and semi-supervised learning remains a challenge. The bottleneck for likelihood-based learning of generative models is the intractable computation of expectations which usually requires expensive Markov chain Monte Carlo (MCMC) sampling, whose convergence can be problematic. The main motivation of the research is to get around this bottleneck. The following are specific aims of the proposed research: (1) Variational optimization of short-run MCMC dynamics for the sampling computations in likelihood-based learning of generative models, by combining the advantages of MCMC and variational inference. (2) Developing biologically plausible generative models with multiple layers of hidden variables, and the associated short-run inference and synthesis dynamics that can account for feedbacks and inhibitions between hidden variables.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.