Robust PAC<inline-formula> <tex-math notation="LaTeX">$^m$</tex-math> </inline-formula> Training Ensemble Models Under Misspecification and Outliers

Matteo Zecchin, Sangwoo Park, Osvaldo Simeone, Marios Kountouris, David Gesbert

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Standard Bayesian learning is known to have suboptimal generalization capabilities under misspecification and in the presence of outliers. Probably approximately correct (PAC)-Bayes theory demonstrates that the free energy criterion minimized by Bayesian learning is a bound on the generalization error for Gibbs predictors (i.e., for single models drawn at random from the posterior) under the assumption of sampling distributions uncontaminated by outliers. This viewpoint provides a justification for the limitations of Bayesian learning when the model is misspecified, requiring ensembling, and when data are affected by outliers. In recent work, PAC-Bayes bounds&#x2014;referred to as PAC<inline-formula> <tex-math notation="LaTeX">$^m$</tex-math> </inline-formula>&#x2014;were derived to introduce free energy metrics that account for the performance of ensemble predictors, obtaining enhanced performance under misspecification. This work presents a novel robust free energy criterion that combines the generalized logarithm score function with PAC<inline-formula> <tex-math notation="LaTeX">$^m$</tex-math> </inline-formula> ensemble bounds. The proposed free energy training criterion produces predictive distributions that are able to concurrently counteract the detrimental effects of misspecification&#x2014;with respect to both likelihood and prior distribution&#x2014;and outliers.

Original languageEnglish (US)
Pages (from-to)1-15
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
StateAccepted/In press - 2023

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence


  • Bayes methods
  • Bayesian learning
  • Europe
  • Pollution measurement
  • Predictive models
  • Robustness
  • Standards
  • Training
  • ensemble models
  • machine learning
  • misspecification
  • outliers
  • robustness


Dive into the research topics of 'Robust PAC<inline-formula> <tex-math notation="LaTeX">$^m$</tex-math> </inline-formula> Training Ensemble Models Under Misspecification and Outliers'. Together they form a unique fingerprint.

Cite this