Ocellus: Highly Parallel Convolution-in-Pixel Scheme Realizing Power-Delay-Efficient Edge Intelligence

With the advent of Edge Intelligence (EI) devices, always-on intelligent and self-powered visual perception systems are receiving considerable attention. These emerging systems require continuous sensing and instant processing; however, the high energy data conversion/transmission of raw data and the limited available energy and computation resources make designing energy-efficient and low bandwidth CMOS vision sensors vital but challenging. This paper proposes a low-power integrated sensing and computing engine, namely Ocellus, which considerably decreases power costs of data movement/conversion and enables data/compute -intensive neural network tasks. Ocellus offers several unique features, including a highly parallel analog convolution-in-pixel scheme and reconfigurable filtering modes with filter pruning capability. These features realize low-precision ternary weight neural networks to mitigate the overhead of analog-to-digital converters and analog buffers. Moreover, the proposed structure supports a zero-skipping scheme to further reduce power consumption. Our circuit-to-application cosimulation results demonstrate comparable, even better, accuracy to the full-precision baseline on object classification tasks, while it achieves a frame rate of 1000 and efficiency of 1.45 TOp/s/W.