CLIP-based Neural Neighbor Style Transfer for 3D Assets

dc.contributor.authorMishra, Shaileshen_US
dc.contributor.authorGranskog, Jonathanen_US
dc.contributor.editorBabaei, Vahiden_US
dc.contributor.editorSkouras, Melinaen_US
dc.date.accessioned2023-05-03T06:02:53Z
dc.date.available2023-05-03T06:02:53Z
dc.date.issued2023
dc.description.abstractWe present a method for transferring the style from a set of images to the texture of a 3D object. The texture of an asset is optimized with a differentiable renderer and losses using pretrained deep neural networks. More specifically, we utilize a nearest-neighbor feature matching (NNFM) loss with CLIP-ResNet50 that we extend to support multiple style images. We improve color accuracy and artistic control with an extra loss on user-provided or automatically extracted color palettes. Finally, we show that a CLIP-based NNFM loss provides a different appearance over a VGG-based one by focusing more on textural details over geometric shapes. However, we note that user preference is still subjective.en_US
dc.description.sectionheadersStylization and Point Clouds
dc.description.seriesinformationEurographics 2023 - Short Papers
dc.identifier.doi10.2312/egs.20231006
dc.identifier.isbn978-3-03868-209-7
dc.identifier.issn1017-4656
dc.identifier.pages25-28
dc.identifier.pages4 pages
dc.identifier.urihttps://doi.org/10.2312/egs.20231006
dc.identifier.urihttps://diglib.eg.org:443/handle/10.2312/egs20231006
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Appearance and texture representations; Rasterization; Supervised learning by regression
dc.subjectComputing methodologies → Appearance and texture representations
dc.subjectRasterization
dc.subjectSupervised learning by regression
dc.titleCLIP-based Neural Neighbor Style Transfer for 3D Assetsen_US
Files
Original bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
025-028.pdf
Size:
58.58 MB
Format:
Adobe Portable Document Format
No Thumbnail Available
Name:
short1013_supplemental.pdf
Size:
24 MB
Format:
Adobe Portable Document Format