Browsing by Author "Kim, Seonghyeon"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    StylePortraitVideo: Editing Portrait Videos with Expression Optimization
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Seo, Kwanggyoon; Oh, Seoung Wug; Lu, Jingwan; Lee, Joon-Young; Kim, Seonghyeon; Noh, Junyong; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
    High-quality portrait image editing has been made easier by recent advances in GANs (e.g., StyleGAN) and GAN inversion methods that project images onto a pre-trained GAN's latent space. However, extending the existing image editing methods, it is hard to edit videos to produce temporally coherent and natural-looking videos. We find challenges in reproducing diverse video frames and preserving the natural motion after editing. In this work, we propose solutions for these challenges. First, we propose a video adaptation method that enables the generator to reconstruct the original input identity, unusual poses, and expressions in the video. Second, we propose an expression dynamics optimization that tweaks the latent codes to maintain the meaningful motion in the original video. Based on these methods, we build a StyleGAN-based high-quality portrait video editing system that can edit videos in the wild in a temporally coherent way at up to 4K resolution.