Subarchitecture Ensemble Pruning in Neural Architecture Search

Yijun Bian, Qingquan Song, Mengnan Du, Jun Yao, Huanhuan Chen, Xia Hu

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Neural architecture search (NAS) is gaining more and more attention in recent years because of its flexibility and remarkable capability to reduce the burden of neural network design. To achieve better performance, however, the searching process usually costs massive computations that might not be affordable for researchers and practitioners. Although recent attempts have employed ensemble learning methods to mitigate the enormous computational cost, however, they neglect a key property of ensemble methods, namely diversity, which leads to collecting more similar subarchitectures with potential redundancy in the final design. To tackle this problem, we propose a pruning method for NAS ensembles called ' subarchitecture ensemble pruning in neural architecture search (SAEP).' It targets to leverage diversity and to achieve subensemble architectures at a smaller size with comparable performance to ensemble architectures that are not pruned. Three possible solutions are proposed to decide which subarchitectures to prune during the searching process. Experimental results exhibit the effectiveness of the proposed method by largely reducing the number of subarchitectures without degrading the performance.

Original languageEnglish (US)
Pages (from-to)7928-7936
Number of pages9
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume33
Issue number12
DOIs
StatePublished - Dec 1 2022
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Keywords

  • Diversity
  • ensemble learning
  • ensemble pruning
  • neural architecture search (NAS)

Fingerprint

Dive into the research topics of 'Subarchitecture Ensemble Pruning in Neural Architecture Search'. Together they form a unique fingerprint.

Cite this