StylePortraitVideo: Editing Portrait Videos with Expression Optimization

Simple item page
dc.contributor.authorSeo, Kwanggyoonen_US
dc.contributor.authorOh, Seoung Wugen_US
dc.contributor.authorLu, Jingwanen_US
dc.contributor.authorLee, Joon-Youngen_US
dc.contributor.authorKim, Seonghyeonen_US
dc.contributor.authorNoh, Junyongen_US
dc.contributor.editorUmetani, Nobuyukien_US
dc.contributor.editorWojtan, Chrisen_US
dc.contributor.editorVouga, Etienneen_US
dc.date.accessioned2022-10-04T06:39:40Z
dc.date.available2022-10-04T06:39:40Z
dc.date.issued2022
dc.description.abstractHigh-quality portrait image editing has been made easier by recent advances in GANs (e.g., StyleGAN) and GAN inversion methods that project images onto a pre-trained GAN's latent space. However, extending the existing image editing methods, it is hard to edit videos to produce temporally coherent and natural-looking videos. We find challenges in reproducing diverse video frames and preserving the natural motion after editing. In this work, we propose solutions for these challenges. First, we propose a video adaptation method that enables the generator to reconstruct the original input identity, unusual poses, and expressions in the video. Second, we propose an expression dynamics optimization that tweaks the latent codes to maintain the meaningful motion in the original video. Based on these methods, we build a StyleGAN-based high-quality portrait video editing system that can edit videos in the wild in a temporally coherent way at up to 4K resolution.en_US
dc.description.number7
dc.description.sectionheadersVideo
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume41
dc.identifier.doi10.1111/cgf.14666
dc.identifier.issn1467-8659
dc.identifier.pages165-175
dc.identifier.pages11 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.14666
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14666
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectCCS Concepts: Computing methodologies --> Computer vision; Image manipulation
dc.subjectComputing methodologies
dc.subjectComputer vision
dc.subjectImage manipulation
dc.titleStylePortraitVideo: Editing Portrait Videos with Expression Optimizationen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
v41i7pp165-175.pdf
Size:
14.89 MB
Format:
Adobe Portable Document Format
Download
Thumbnail Image
Name:
paper1038_supp.pdf
Size:
6.87 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
paper1038_supp.zip
Size:
42.3 MB
Format:
Zip file
Download
Collections
41-Issue 7