Compressed Particle-Based Federated Bayesian Learning and Unlearning

Jinu Gong; Osvaldo Simeone; Joonhyuk Kang

doi:10.1109/LCOMM.2022.3223655

Compressed Particle-Based Federated Bayesian Learning and Unlearning

Jinu Gong, Osvaldo Simeone, Joonhyuk Kang

Research output: Contribution to journal › Article › peer-review

Abstract

Conventional frequentist federated learning (FL) schemes are known to yield overconfident decisions. Bayesian FL addresses this issue by allowing agents to process and exchange uncertainty information encoded in distributions over the model parameters. However, this comes at the cost of a larger per-iteration communication overhead. This letter investigates whether Bayesian FL can still provide advantages in terms of calibration when constraining communication bandwidth. We present compressed particle-based Bayesian FL protocols for FL and federated “unlearning" that apply quantization and sparsification across multiple particles. The experimental results confirm that the benefits of Bayesian FL are robust to bandwidth constraints.

Original language	English (US)
Pages (from-to)	1
Number of pages	1
Journal	IEEE Communications Letters
DOIs	https://doi.org/10.1109/LCOMM.2022.3223655
State	Accepted/In press - 2022
Externally published	Yes

All Science Journal Classification (ASJC) codes

Modeling and Simulation
Computer Science Applications
Electrical and Electronic Engineering

Keywords

Bayes methods
Bayesian learning
Federated learning
Federated learning
Machine unlearning
Protocols
Quantization (signal)
Servers
Stein variational gradient descent
Training
Uncertainty
Wireless communication

Access to Document

10.1109/LCOMM.2022.3223655

Cite this

@article{bcb10f2bd84b45679f2a1db559bcc805,

title = "Compressed Particle-Based Federated Bayesian Learning and Unlearning",

abstract = "Conventional frequentist federated learning (FL) schemes are known to yield overconfident decisions. Bayesian FL addresses this issue by allowing agents to process and exchange uncertainty information encoded in distributions over the model parameters. However, this comes at the cost of a larger per-iteration communication overhead. This letter investigates whether Bayesian FL can still provide advantages in terms of calibration when constraining communication bandwidth. We present compressed particle-based Bayesian FL protocols for FL and federated “unlearning{"} that apply quantization and sparsification across multiple particles. The experimental results confirm that the benefits of Bayesian FL are robust to bandwidth constraints.",

keywords = "Bayes methods, Bayesian learning, Federated learning, Federated learning, Machine unlearning, Protocols, Quantization (signal), Servers, Stein variational gradient descent, Training, Uncertainty, Wireless communication",

author = "Jinu Gong and Osvaldo Simeone and Joonhyuk Kang",

note = "Publisher Copyright: IEEE",

year = "2022",

doi = "10.1109/LCOMM.2022.3223655",

language = "English (US)",

pages = "1",

journal = "IEEE Communications Letters",

issn = "1089-7798",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Compressed Particle-Based Federated Bayesian Learning and Unlearning

AU - Gong, Jinu

AU - Simeone, Osvaldo

AU - Kang, Joonhyuk

N1 - Publisher Copyright: IEEE

PY - 2022

Y1 - 2022

N2 - Conventional frequentist federated learning (FL) schemes are known to yield overconfident decisions. Bayesian FL addresses this issue by allowing agents to process and exchange uncertainty information encoded in distributions over the model parameters. However, this comes at the cost of a larger per-iteration communication overhead. This letter investigates whether Bayesian FL can still provide advantages in terms of calibration when constraining communication bandwidth. We present compressed particle-based Bayesian FL protocols for FL and federated “unlearning" that apply quantization and sparsification across multiple particles. The experimental results confirm that the benefits of Bayesian FL are robust to bandwidth constraints.

AB - Conventional frequentist federated learning (FL) schemes are known to yield overconfident decisions. Bayesian FL addresses this issue by allowing agents to process and exchange uncertainty information encoded in distributions over the model parameters. However, this comes at the cost of a larger per-iteration communication overhead. This letter investigates whether Bayesian FL can still provide advantages in terms of calibration when constraining communication bandwidth. We present compressed particle-based Bayesian FL protocols for FL and federated “unlearning" that apply quantization and sparsification across multiple particles. The experimental results confirm that the benefits of Bayesian FL are robust to bandwidth constraints.

KW - Bayes methods

KW - Bayesian learning

KW - Federated learning

KW - Machine unlearning

KW - Protocols

KW - Quantization (signal)

KW - Servers

KW - Stein variational gradient descent

KW - Training

KW - Uncertainty

KW - Wireless communication

UR - http://www.scopus.com/inward/record.url?scp=85144060425&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85144060425&partnerID=8YFLogxK

U2 - 10.1109/LCOMM.2022.3223655

DO - 10.1109/LCOMM.2022.3223655

M3 - Article

AN - SCOPUS:85144060425

SN - 1089-7798

SP - 1

JO - IEEE Communications Letters

JF - IEEE Communications Letters

ER -

Compressed Particle-Based Federated Bayesian Learning and Unlearning

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this