EG 2023 - Short Papers

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/2633300

Browse

Browsing EG 2023 - Short Papers by Subject "Computing methodologies → Appearance and texture representations"

Now showing 1 - 1 of 1
  • No Thumbnail Available
    Item
    CLIP-based Neural Neighbor Style Transfer for 3D Assets
    (The Eurographics Association, 2023) Mishra, Shailesh; Granskog, Jonathan; Babaei, Vahid; Skouras, Melina
    We present a method for transferring the style from a set of images to the texture of a 3D object. The texture of an asset is optimized with a differentiable renderer and losses using pretrained deep neural networks. More specifically, we utilize a nearest-neighbor feature matching (NNFM) loss with CLIP-ResNet50 that we extend to support multiple style images. We improve color accuracy and artistic control with an extra loss on user-provided or automatically extracted color palettes. Finally, we show that a CLIP-based NNFM loss provides a different appearance over a VGG-based one by focusing more on textural details over geometric shapes. However, we note that user preference is still subjective.