Semantic Stylization and Shading via Segmentation Atlas utilizing Deep Learning Approaches

Loading...
Thumbnail Image
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
We present a novel hybrid approach for semantic stylization of surface materials of 3D models while preserving shading. Based on a hybrid approach that builds on directly applying style transfer on the object surface obtained by learning-based or traditional methods such as 3D scanners or structured light systems, thereby overcoming artifacts like halos, ghosting or lacking quality of the geometric representation produced by other 3D stylization methods. For this purpose, our methods involves (i) the initial generation of a segmentation map parameterized over the object surface inferred based on a deep-learning-based foundation model to guide the stylization and shading of different regions of the 3D model, and (ii) a subsequent 2D style transfer that allows the exchange or stylization of surface materials in high quality. By delivering high-quality semantic perceptive reconstructions in a shorter timeframe than current approaches using manual 3D segmentation and stylization, our approach holds significant potential for various application scenarios including creative design, architecture and cultural heritage.
Description

CCS Concepts: Computing methodologies → Computer graphics; Reflectance modeling; Image manipulation; Scene understanding

        
@inproceedings{
10.2312:stag.20241352
, booktitle = {
Smart Tools and Applications in Graphics - Eurographics Italian Chapter Conference
}, editor = {
Caputo, Ariel
and
Garro, Valeria
and
Giachetti, Andrea
and
Castellani, Umberto
and
Dulecha, Tinsae Gebrechristos
}, title = {{
Semantic Stylization and Shading via Segmentation Atlas utilizing Deep Learning Approaches
}}, author = {
Sinha, Saptarshi Neil
and
Kühn, Paul Julius
and
Rojtberg, Pavel
and
Graf, Holger
and
Kuijper, Arjan
and
Weinmann, Michael
}, year = {
2024
}, publisher = {
The Eurographics Association
}, ISSN = {
2617-4855
}, ISBN = {
978-3-03868-265-3
}, DOI = {
10.2312/stag.20241352
} }
Citation