The Challenges of Relighting from Multi-View Observations

No Thumbnail Available
Files
mam20241181.pdf (9.53 MB)
Date
2024
Authors
Siersleben, Daniel
Ortiz-Cayon, Rodrigo
Istenic, Klemen
Tomoto, Yusuke
Schaub-Meyer, Simone
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
High-quality geometry reconstruction from multi-view images with subsequent appearance decomposition into the physical shading components could enable a seamless integration of neural reconstructions into the modern rendering workflow. While 3D reconstruction techniques have steadily improved, the task of inverse rendering by decomposing an appearance into lighting effects and material properties remains fundamentally ill-posed and highly ambiguous. We show that current state-of-the-art inverse rendering approaches fail to accurately recover material properties, significantly impacting relighting quality. Furthermore, we demonstrate that existing evaluation methods, which rely on image-based metrics, do not adequately capture the reconstruction quality in novel lighting conditions. Our findings illustrate the dependence of current systems on simplified setups with predefined illumination, which are necessary to reliably disentangle light and material contributions and to ultimately achieve convincing relighting.
Description

        
@inproceedings{
10.2312:mam.20241181
,
booktitle = {
Workshop on Material Appearance Modeling
Joint MAM - MANER Conference - Material Appearance Network for Education and Research
},
editor = {
Hardeberg, Jon Yngve
 and 
Rushmeier, Holly
},
title = {{
The Challenges of Relighting from Multi-View Observations
}},
author = {
Siersleben, Daniel
 and 
Ortiz-Cayon, Rodrigo
 and 
Istenic, Klemen
 and 
Tomoto, Yusuke
 and 
Schaub-Meyer, Simone
},
year = {
2024
},
publisher = {
The Eurographics Association
},
ISSN = {
2309-5059
},
ISBN = {
978-3-03868-264-6
},
DOI = {
10.2312/mam.20241181
}
}

@inproceedings{
10.2312:mam.20241181
,
booktitle = {
Workshop on Material Appearance Modeling
Joint MAM - MANER Conference - Material Appearance Network for Education and Research
},
editor = {
Hardeberg, Jon Yngve
 and 
Rushmeier, Holly
},
title = {{
The Challenges of Relighting from Multi-View Observations
}},
author = {
Siersleben, Daniel
 and 
Ortiz-Cayon, Rodrigo
 and 
Istenic, Klemen
 and 
Tomoto, Yusuke
 and 
Schaub-Meyer, Simone
},
year = {
2024
},
publisher = {
The Eurographics Association
},
ISSN = {
2309-5059
},
ISBN = {
978-3-03868-264-6
},
DOI = {
10.2312/mam.20241181
}
}
Citation
URI
https://doi.org/10.2312/mam.20241181
 https://diglib.eg.org/handle/10.2312/mam20241181
Collections
MAM2024 - MANER Conference London 2024