TSDN: Transport-based Stylization for Dynamic NeRF
Loading...
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
While previous Neural Radiance Fields (NeRF) stylization methods achieve visually appealing results on transferring color style for static NeRF scenes, they lack the ability to stylize dynamic NeRF scenes with geometrically stylized features (like brushstrokes or feature elements from artists' works), which is also important for style transfer. However, directly stylizing each frame of dynamic NeRF independently with geometrically stylized features would lead to flickering results due to bad feature alignment. To overcome these problems, in this paper, we propose Transport-based Stylization for Dynamic NeRF (TSDN), a new dynamic NeRF stylization method that is able to stylize geometric features and align them with the motion in the scene. TSDN utilizes stylization guiding velocity fields to advect dynamic NeRF to get stylized results and then transfers these velocity fields between frames to maintain feature alignment. Also, to deal with the noisy stylized results due to the ambiguity of the deformation field, we propose a feature advection scheme and a novel regularization function specified for dynamic NeRF. The experiment results show that our method has the ability to stylize dynamic scenes with detailed geometrically stylized features from videos or multi-view image inputs, while preserving the original color style if desired. This capability is not present in previous video stylization methods.
Description
CCS Concepts: Computing methodologies → Image processing; Volumetric models; Shape representations
@inproceedings{10.2312:pg.20241300,
booktitle = {Pacific Graphics Conference Papers and Posters},
editor = {Chen, Renjie and Ritschel, Tobias and Whiting, Emily},
title = {{TSDN: Transport-based Stylization for Dynamic NeRF}},
author = {Gong, Yuning and Song, Mingqing and Ren, Xiaohua and Liao, Yuanjun and Zhang, Yanci},
year = {2024},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-250-9},
DOI = {10.2312/pg.20241300}
}