Calibration-Aided Edge Inference Offloading via Adaptive Model Partitioning of Deep Neural Networks

Roberto G. Pacheco, Rodrigo S. Couto, Osvaldo Simeone

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Mobile devices can offload deep neural network (DNN)-based inference to the cloud, overcoming local hardware and energy limitations. However, offloading adds communication delay, thus increasing the overall inference time, and hence it should be used only when needed. An approach to address this problem consists of the use of adaptive model partitioning based on early-exit DNNs. Accordingly, the inference starts at the mobile device, and an intermediate layer estimates the accuracy: If the estimated accuracy is sufficient, the device takes the inference decision; Otherwise, the remaining layers of the DNN run at the cloud. Thus, the device offloads the inference to the cloud only if it cannot classify a sample with high confidence. This offloading requires a correct accuracy prediction at the device. Nevertheless, DNNs are typically miscalibrated, providing overconfident decisions. This work shows that the employment of a miscalibrated early-exit DNN for offloading via model partitioning can significantly decrease inference accuracy. In contrast, we argue that implementing a calibration algorithm prior to deployment can solve this problem, allowing for more reliable offloading decisions.

Original languageEnglish (US)
Title of host publicationICC 2021 - IEEE International Conference on Communications, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728171227
DOIs
StatePublished - Jun 2021
Externally publishedYes
Event2021 IEEE International Conference on Communications, ICC 2021 - Virtual, Online, Canada
Duration: Jun 14 2021Jun 23 2021

Publication series

NameIEEE International Conference on Communications
ISSN (Print)1550-3607

Conference

Conference2021 IEEE International Conference on Communications, ICC 2021
Country/TerritoryCanada
CityVirtual, Online
Period6/14/216/23/21

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Calibration-Aided Edge Inference Offloading via Adaptive Model Partitioning of Deep Neural Networks'. Together they form a unique fingerprint.

Cite this