Visual Relationship Detection: A Survey

Jun Cheng, Lei Wang, Jiaji Wu, Xiping Hu, Gwanggil Jeon, Dacheng Tao, Mengchu Zhou

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


Visual relationship detection (VRD) is one newly developed computer vision task, aiming to recognize relations or interactions between objects in an image. It is a further learning task after object recognition, and is important for fully understanding images even the visual world. It has numerous applications, such as image retrieval, machine vision in robotics, visual question answer (VQA), and visual reasoning. However, this problem is difficult since relationships are not definite, and the number of possible relations is much larger than objects. So the complete annotation for visual relationships is much more difficult, making this task hard to learn. Many approaches have been proposed to tackle this problem especially with the development of deep neural networks in recent years. In this survey, we first introduce the background of visual relations. Then, we present categorization and frameworks of deep learning models for visual relationship detection. The high-level applications, benchmark datasets, as well as empirical analysis are also introduced for comprehensive understanding of this task.

Original languageEnglish (US)
Pages (from-to)8453-8466
Number of pages14
JournalIEEE Transactions on Cybernetics
Issue number8
StatePublished - Aug 1 2022

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering


  • Deep learning
  • detection
  • neural networks
  • visual relation


Dive into the research topics of 'Visual Relationship Detection: A Survey'. Together they form a unique fingerprint.

Cite this