High-Resolution Neural Face Swapping for Visual Effects

Simple item page
dc.contributor.authorNaruniec, Jaceken_US
dc.contributor.authorHelminger, Leonharden_US
dc.contributor.authorSchroers, Christopheren_US
dc.contributor.authorWeber, Romann M.en_US
dc.contributor.editorDachsbacher, Carsten and Pharr, Matten_US
dc.date.accessioned2020-06-28T15:25:03Z
dc.date.available2020-06-28T15:25:03Z
dc.date.issued2020
dc.description.abstractIn this paper, we propose an algorithm for fully automatic neural face swapping in images and videos. To the best of our knowledge, this is the first method capable of rendering photo-realistic and temporally coherent results at megapixel resolution. To this end, we introduce a progressively trained multi-way comb network and a light- and contrast-preserving blending method. We also show that while progressive training enables generation of high-resolution images, extending the architecture and training data beyond two people allows us to achieve higher fidelity in generated expressions. When compositing the generated expression onto the target face, we show how to adapt the blending strategy to preserve contrast and low-frequency lighting. Finally, we incorporate a refinement strategy into the face landmark stabilization algorithm to achieve temporal stability, which is crucial for working with high-resolution videos. We conduct an extensive ablation study to show the influence of our design choices on the quality of the swap and compare our work with popular state-of-the-art methods.en_US
dc.description.number4
dc.description.sectionheadersImages and Textures
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume39
dc.identifier.doi10.1111/cgf.14062
dc.identifier.issn1467-8659
dc.identifier.pages173-184
dc.identifier.urihttps://doi.org/10.1111/cgf.14062
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14062
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectComputing methodologies
dc.subjectImage manipulation
dc.subjectUnsupervised learning
dc.subjectNeural networks
dc.titleHigh-Resolution Neural Face Swapping for Visual Effectsen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
v39i4pp173-184.pdf
Size:
29.85 MB
Format:
Adobe Portable Document Format
Download
Thumbnail Image
Name:
appendix.pdf
Size:
3.23 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
egsr2020.mp4
Size:
297.49 MB
Format:
Unknown data format
Download
Collections
39-Issue 4