A Lightweight Block With Information Flow Enhancement for Convolutional Neural Networks

Zhiqiang Bao, Shunzhi Yang, Zhenhua Huang, Meng Chu Zhou, Yunwen Chen

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Convolutional neural networks (CNNs) have demonstrated excellent capability in various visual recognition tasks but impose an excessive computational burden. The latter problem is commonly solved by utilizing lightweight sparse networks. However, such networks have a limited receptive field in a few layers, and the majority of these networks face a severe information barrage due to their sparse structures. Spurred by these deficiencies, this work proposes a Squeeze Convolution block with Information Flow Enhancement (SCIFE), comprising a Divide-and-Squeeze Convolution and an Information Flow Enhancement scheme. The former module constructs a multi-layer structure through multiple squeeze operations to increase the receptive field and reduce computation. The latter replaces the affine transformation with the point convolution and dynamically adjusts the activation function's threshold, enhancing information flow in both channels and layers. Moreover, we reveal that the original affine transformation may harm the network's generalization capability. To overcome this issue, we utilize a point convolution with a zero initial mean. SCIFE can serve as a plug-and-play replacement for vanilla convolution blocks in mainstream CNNs, while extensive experimental results demonstrate that CNNs equipped with SCIFE compress benchmark structures without sacrificing performance, outperforming their competitors.

Original languageEnglish (US)
Pages (from-to)3570-3584
Number of pages15
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume33
Issue number8
DOIs
StatePublished - Aug 1 2023

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Media Technology

Keywords

  • Convolutional neural network
  • activation function
  • affine transformation
  • information flow
  • lightweight

Fingerprint

Dive into the research topics of 'A Lightweight Block With Information Flow Enhancement for Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this