Dyna-Optics: Architecting a Channel-Adaptive DNN Near-Sensor Optical Accelerator for Dynamic Inference

This paper presents a high-performance and energy-efficient near-sensor optical Deep Neural Network (DNN) accelerator - named Dyna-Optics - for dynamic inference in vision applications. Dyna-Optics leverages the efficiency of silicon photonic devices in an innovative real-time adjustable architecture supported by a novel channel-adaptive dynamic neural network algorithm to perform near-sensor granularity-controllable convolution operations for the first time. Dyna-Optics is co-designed to adjust its photonic device allocations and computing path through a novel device arm-dropping mechanism to best align varying workloads by eliminating the humongous energy consumption imposed by the weight tuning on photonic devices. Our device-to-architecture simulation results demonstrate that Dyna-Optics enables real-time trade-offs between speed, energy, and accuracy after model deployment. It can process ∼84 Kilo FPS/W with slight accuracy degradation, reducing power consumption by a factor of up to ∼6.1× and 52× on average compared with existing photonic accelerators and GPU baselines.