Online statistical inference for parameters estimation with linear-equality constraints

Ruiqi Liu, Mingao Yuan, Zuofeng Shang

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Stochastic gradient descent (SGD) and projected stochastic gradient descent (PSGD) are scalable algorithms to compute model parameters in unconstrained and constrained optimization problems. In comparison with SGD, PSGD forces its iterative values into the constrained parameter space via projection. From a statistical point of view, this paper studies the limiting distribution of PSGD-based estimate when the true parameters satisfy some linear-equality constraints. Our theoretical findings reveal the role of projection played in the uncertainty of the PSGD-based estimate. As a byproduct, we propose an online hypothesis testing procedure to test the linear-equality constraints. Simulation studies on synthetic data and an application to a real-world dataset confirm our theory.

Original languageEnglish (US)
Article number105017
JournalJournal of Multivariate Analysis
Volume191
DOIs
StatePublished - Sep 2022

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Numerical Analysis
  • Statistics, Probability and Uncertainty

Keywords

  • Constrained optimization
  • Online inference
  • Projected stochastic gradient descent algorithm

Fingerprint

Dive into the research topics of 'Online statistical inference for parameters estimation with linear-equality constraints'. Together they form a unique fingerprint.

Cite this