MOGAN: Morphologic-Structure-Aware Generative Learning From a Single Image

Jinshu Chen, Qihui Xu, Qi Kang, Meng Chu Zhou

Research output: Contribution to journalArticlepeer-review

Abstract

In most interactive image generation tasks, given regions of interest (ROI) by users, the generated results are expected to have adequate diversities in appearance while maintaining correct and reasonable structures in original images. Such tasks become more challenging if only limited data is available. Recently proposed generative models complete training based on only one image. They pay much attention to the monolithic feature of the sample while ignoring the actual semantic information of different objects inside the sample. As a result, for ROI-based generation tasks, they may produce inappropriate samples with excessive randomicity and without maintaining the related objects’ correct structures. To address this issue, this work introduces a morphologic-structure-aware generative adversarial network named MOGAN that produces random samples with diverse appearances and reliable structures based on only one image. For training for ROI, we propose to utilize the data coming from the original image being augmented and bring in a novel module to transform such augmented data into knowledge containing both structures and appearances, thus enhancing the model’s comprehension of the sample. To learn the rest areas other than ROI, we employ binary masks to ensure the generation isolated from ROI. Finally, we set parallel and hierarchical branches of the mentioned learning process. Compared with other single image generative adversarial network schemes, our approach focuses on internal features, including the maintenance of rational structures and variation on appearance. Experiments confirm a better capacity of our model on ROI-based image generation tasks than its competitive peers.

Original languageEnglish (US)
Pages (from-to)1-13
Number of pages13
JournalIEEE Transactions on Systems, Man, and Cybernetics: Systems
DOIs
StateAccepted/In press - 2023
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Keywords

  • Feature extraction
  • Generative adversarial networks
  • Generative adversarial networks (GANs)
  • Image synthesis
  • Periodic structures
  • Semantics
  • Task analysis
  • Training
  • machine learning
  • morphologic awareness
  • regions of interest (ROI)-based image generation tasks
  • single sample

Fingerprint

Dive into the research topics of 'MOGAN: Morphologic-Structure-Aware Generative Learning From a Single Image'. Together they form a unique fingerprint.

Cite this