Wireless Federated Langevin Monte Carlo: Repurposing Channel Noise for Bayesian Sampling and Privacy

Dongzhu Liu, Osvaldo Simeone

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Most works on federated learning (FL) focus on the most common frequentist formulation of learning whereby the goal is minimizing the global empirical loss. Frequentist learning, however, is known to be problematic in the regime of limited data as it fails to quantify epistemic uncertainty in prediction. Bayesian learning provides a principled solution to this problem by shifting the optimization domain to the space of distribution in the model parameters. This paper proposes a novel mechanism for the efficient implementation of Bayesian learning in wireless systems. Specifically, we focus on a standard gradient-based Markov Chain Monte Carlo (MCMC) method, namely Langevin Monte Carlo (LMC), and we introduce a novel protocol, termed Wireless Federated LMC (WFLMC), that is able to repurpose channel noise for the double role of seed randomness for MCMC sampling and of privacy preservation. To this end, based on the analysis of the Wasserstein distance between sample distribution and global posterior distribution under privacy and power constraints, we introduce a power allocation strategy as the solution of a convex program. The analysis identifies distinct operating regimes in which the performance of the system is power-limited, privacy-limited, or limited by the requirement of MCMC sampling. Both analytical and simulation results demonstrate that, if the channel noise is properly accounted for under suitable conditions, it can be fully repurposed for both MCMC sampling and privacy preservation, obtaining the same performance as in an ideal communication setting that is not subject to privacy constraints.

Original languageEnglish (US)
Pages (from-to)2946-2961
Number of pages16
JournalIEEE Transactions on Wireless Communications
Volume22
Issue number5
DOIs
StatePublished - May 1 2023

All Science Journal Classification (ASJC) codes

  • Applied Mathematics
  • Electrical and Electronic Engineering
  • Computer Science Applications

Keywords

  • Federated learning~(FL)
  • differential privacy~(DP)
  • langevin monte carlo~(LMC)
  • over-the-air computation
  • power allocation
  • scheduling
  • uncoded transmission

Fingerprint

Dive into the research topics of 'Wireless Federated Langevin Monte Carlo: Repurposing Channel Noise for Bayesian Sampling and Privacy'. Together they form a unique fingerprint.

Cite this