Makeup Extraction of 3D Representation via Illumination-Aware Image Decomposition

Simple item page
dc.contributor.authorYang, Xingchaoen_US
dc.contributor.authorTaketomi, Takafumien_US
dc.contributor.authorKanamori, Yoshihiroen_US
dc.contributor.editorMyszkowski, Karolen_US
dc.contributor.editorNiessner, Matthiasen_US
dc.date.accessioned2023-05-03T06:10:21Z
dc.date.available2023-05-03T06:10:21Z
dc.date.issued2023
dc.description.abstractFacial makeup enriches the beauty of not only real humans but also virtual characters; therefore, makeup for 3D facial models is highly in demand in productions. However, painting directly on 3D faces and capturing real-world makeup are costly, and extracting makeup from 2D images often struggles with shading effects and occlusions. This paper presents the first method for extracting makeup for 3D facial models from a single makeup portrait. Our method consists of the following three steps. First, we exploit the strong prior of 3D morphable models via regression-based inverse rendering to extract coarse materials such as geometry and diffuse/specular albedos that are represented in the UV space. Second, we refine the coarse materials, which may have missing pixels due to occlusions. We apply inpainting and optimization. Finally, we extract the bare skin, makeup, and an alpha matte from the diffuse albedo. Our method offers various applications for not only 3D facial models but also 2D portrait images. The extracted makeup is well-aligned in the UV space, from which we build a large-scale makeup dataset and a parametric makeup model for 3D faces. Our disentangled materials also yield robust makeup transfer and illumination-aware makeup interpolation/removal without a reference image.en_US
dc.description.number2
dc.description.sectionheadersFaces
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume42
dc.identifier.doi10.1111/cgf.14762
dc.identifier.issn1467-8659
dc.identifier.pages293-307
dc.identifier.pages15 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.14762
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14762
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectCCS Concepts: Computing methodologies -> Computer graphics; Computer vision; Machine learning
dc.subjectComputing methodologies
dc.subjectComputer graphics
dc.subjectComputer vision
dc.subjectMachine learning
dc.titleMakeup Extraction of 3D Representation via Illumination-Aware Image Decompositionen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
v42i2pp293-307_cgf14762.pdf
Size:
4.02 MB
Format:
Adobe Portable Document Format
Download
Collections
42-Issue 2