VideoMat: Extracting PBR Materials from Video Diffusion Models
Loading...
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association and John Wiley & Sons Ltd.
Abstract
We leverage finetuned video diffusion models, intrinsic decomposition of videos, and physically-based differentiable rendering to generate high quality materials for 3D models given a text prompt or a single image. We condition a video diffusion model to respect the input geometry and lighting condition. This model produces multiple views of a given 3D model with coherent material properties. Secondly, we use a recent model to extract intrinsics (base color, roughness, metallic) from the generated video. Finally, we use the intrinsics alongside the generated video in a differentiable path tracer to robustly extract PBR materials directly compatible with common content creation tools.
Description
CCS Concepts: Computing methodologies → Appearance and texture representations; Ray tracing; Reflectance modeling
@article{10.1111:cgf.70180,
journal = {Computer Graphics Forum},
title = {{VideoMat: Extracting PBR Materials from Video Diffusion Models}},
author = {Munkberg, Jacob and Wang, Zian and Liang, Ruofan and Shen, Tianchang and Hasselgren, Jon},
year = {2025},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.70180}
}