3D-CariNet: End-to-end 3D Caricature Generation from Natural Face Images with Differentiable Renderer
Loading...
Date
2021
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Caricatures are an artistic representation of human faces to express satire and humor. Caricature generation of human faces is a hotspot in CG research. Previous work mainly focuses on 2D caricatures generation from face photos or 3D caricature reconstruction from caricature images. In this paper, we propose a novel end-to-end method to directly generate personalized 3D caricatures from a single natural face image. It can create not only exaggerated geometric shapes, but also heterogeneous texture styles. Firstly, we construct a synthetic dataset containing matched data pairs composed of face photos, caricature images, and 3D caricatures. Then, we design a graph convolutional autoencoder to build a non-linear colored mesh model to learn the shape and texture of 3D caricatures. To make the network end-to-end trainable, we incorporate a differentiable renderer to render 3D caricatures into caricature images inversely. Experiments demonstrate that our method can achieve 3D caricature generation with various texture styles from face images while maintaining personality characteristics.
Description
@inproceedings{10.2312:pg.20211387,
booktitle = {Pacific Graphics Short Papers, Posters, and Work-in-Progress Papers},
editor = {Lee, Sung-Hee and Zollmann, Stefanie and Okabe, Makoto and Wünsche, Burkhard},
title = {{3D-CariNet: End-to-end 3D Caricature Generation from Natural Face Images with Differentiable Renderer}},
author = {Huang, Meijia and Dai, Ju and Pan, Junjun and Bai, Junxuan and Qin, Hong},
year = {2021},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-162-5},
DOI = {10.2312/pg.20211387}
}