CNS Core: Small: Toward Opportunistic, Fast, and Robust In-Cache AI Acceleration at the Edge

Internet of Things (IoT) is anticipated to comprise approximately 75+ billion interconnected devices by 2025. Many Artificial Intelligence (AI)-enabled IoT devices consist of sensory imaging systems that enable data collection from people and the environment. However, insufficient computing ability of small IoT devices such as smartphones, wearable devices, etc., and memory/compute-intensive AI tasks prevent AI techniques from being widely deployed in such devices. This proposal enables a smooth transition from the state-of-the-art cloud-centric IoT approach to a data-centric approach, enabling mobile edge devices to perform computation close to the sensor by repurposing the cache memory to a data-parallel processing unit. This will remarkably reduce the power consumption and latency of data transmission to the cloud. Moreover, this project seeks to design and deploy new hardware-oriented AI algorithms into edge devices for efficient image processing, reducing the computation complexity and memory access, while maintaining accuracy. With the technologies developed in this project, more powerful and stable IoT devices can be introduced ensuring accelerated operation with applications of societal importance spanning healthcare monitoring, automotive applications, industrial and agriculture sensing, intelligent infrastructure, etc. A comprehensive circuit-to-system assessment framework will be adopted to systematically evaluate the performance of the system on several IoT workload suites. This project will make a strong effort on developing undergraduate and graduate course modules, propagating transportable and open-source models, and broadening STEM participation through publications/presentations at conferences and workshops and involving undergraduate minority students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.