A Domain-Guided Noise-Optimization-Based Inversion Method for Facial Image Manipulation

Nan Yang, Zeyu Zheng, Mengchu Zhou, Xiwang Guo, Liang Qi, Tianran Wang

Research output: Contribution to journalArticlepeer-review

Abstract

A style-based architecture (StyleGAN2) yields outstanding results in data-driven unconditional generative image modeling. This work proposes a Domain-guided Noise-optimization-based Inversion (DNI) method to perform facial image manipulation. It works based on an inverse code that includes: 1) a novel domain-guided encoder called Image2latent to project the image to StyleGAN2 latent space, which can reconstruct an input image with high-quality and maintain its semantic meaning well; 2) a noise optimization mechanism in which a set of noise vectors are used to capture the high-frequency details such as image edges, further improving image reconstruction quality; and 3) a mask for seamless image fusion and local style migration. We further propose a novel semantic alignment evaluation pipeline. It evaluates the semantic alignment with an inverse code by using different attribute boundaries. Extensive qualitative and quantitative comparisons show that DNI can capture rich semantic information and achieve a satisfactory image reconstruction. It can realize a variety of facial image manipulation tasks and outperform state of the art.

Original languageEnglish (US)
Article number9462525
Pages (from-to)6198-6211
Number of pages14
JournalIEEE Transactions on Image Processing
Volume30
DOIs
StatePublished - 2021

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design

Keywords

  • Deep learning
  • domain-guided encoder
  • generative adversarial networks
  • noise optimization

Fingerprint

Dive into the research topics of 'A Domain-Guided Noise-Optimization-Based Inversion Method for Facial Image Manipulation'. Together they form a unique fingerprint.

Cite this