TY - JOUR
T1 - Optimizing Pipelined Computation and Communication for Latency-Constrained Edge Learning
AU - Skatchkovsky, Nicolas
AU - Simeone, Osvaldo
N1 - Funding Information:
Manuscript received May 13, 2019; accepted June 8, 2019. Date of publication June 13, 2019; date of current version September 10, 2019. The authors have received funding from the European Research Council (ERC) under the European Union Horizon 2020 Research and Innovation Programme (Grant Agreement No. 725731). The associate editor coordinating the review of this letter and approving it for publication was E. Bastug. (Corresponding author: Nicolas Skatchkovsky.) The authors are with the Department of Informatics, King’s College London, London WC2R 2LS, U.K. (e-mail: nicolas.skatchkovsky@kcl.ac.uk; osvaldo.simeone@kcl.ac.uk). Digital Object Identifier 10.1109/LCOMM.2019.2922658 Fig. 1. An edge computing system, in which training of a model parametrized by vector w takes place at an edge processor based on data received from a device using a protocol with timeline illustrated in Fig. 2 (OH = overhead).
Publisher Copyright:
© 1997-2012 IEEE.
PY - 2019/9
Y1 - 2019/9
N2 - Consider a device that is connected to an edge processor via a communication channel. The device holds local data that is to be offloaded to the edge processor so as to train a machine learning model, e.g., for regression or classification. Transmission of the data to the learning processor, as well as training based on stochastic gradient descent (SGD), must be both completed within a time limit. Assuming that communication and computation can be pipelined, this letter investigates the optimal choice for the packet payload size, given the overhead of each data packet transmission and the ratio between the computation and the communication rates. This amounts to a tradeoff between bias and variance, since communicating the entire data set first reduces the bias of the training process but it may not leave sufficient time for learning. Analytical bounds on the expected optimality gap are derived so as to enable an effective optimization, which is validated in numerical results.
AB - Consider a device that is connected to an edge processor via a communication channel. The device holds local data that is to be offloaded to the edge processor so as to train a machine learning model, e.g., for regression or classification. Transmission of the data to the learning processor, as well as training based on stochastic gradient descent (SGD), must be both completed within a time limit. Assuming that communication and computation can be pipelined, this letter investigates the optimal choice for the packet payload size, given the overhead of each data packet transmission and the ratio between the computation and the communication rates. This amounts to a tradeoff between bias and variance, since communicating the entire data set first reduces the bias of the training process but it may not leave sufficient time for learning. Analytical bounds on the expected optimality gap are derived so as to enable an effective optimization, which is validated in numerical results.
KW - Machine learning
KW - mobile edge computing
KW - stochastic gradient descent
UR - http://www.scopus.com/inward/record.url?scp=85072262620&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85072262620&partnerID=8YFLogxK
U2 - 10.1109/LCOMM.2019.2922658
DO - 10.1109/LCOMM.2019.2922658
M3 - Article
AN - SCOPUS:85072262620
SN - 1089-7798
VL - 23
SP - 1542
EP - 1546
JO - IEEE Communications Letters
JF - IEEE Communications Letters
IS - 9
M1 - 8736251
ER -