42-Issue 7

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/3543886

Browse

Now showing 1 - 20 of 57

Dissection Puzzles Composed of Multicolor Polyominoes
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Kita, Naoki; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Dissection puzzles leverage geometric dissections, wherein a set of puzzle pieces can be reassembled in various configurations to yield unique geometric figures. Mathematically, a dissection between two 2D polygons can always be established. Consequently, researchers and puzzle enthusiasts strive to design unique dissection puzzles using the fewest pieces feasible. In this study, we introduce novel dissection puzzles crafted with multi-colored polyominoes. Diverging from the traditional aim of establishing geometric dissection between two 2D polygons with the minimal piece count, we seek to identify a common pool of polyomino pieces with colored faces that can be configured into multiple distinct shapes and appearances. Moreover, we offer a method to identify an optimized sequence for rearranging pieces from one form to another, thus minimizing the total relocation distance. This approach can guide users in puzzle assembly and lessen their physical exertion when manually reconfiguring pieces. It could potentially also decrease power consumption when pieces are reorganized using robotic assistance. We showcase the efficacy of our proposed approach through a wide range of shapes and appearances.
Robust Distribution-aware Color Correction for Single-shot Images
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Dhillon, Daljit Singh J.; Joshi, Parisha; Baron, Jessica; Patterson, Eric K.; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Color correction for photographed images is an ill-posed problem. It is also a crucial initial step towards material acquisition for inverse rendering methods or pipelines. Several state-of-the-art methods rely on reducing color differences for imaged reference color chart blocks of known color values to devise or optimize their solution. In this paper, we first establish through simulations the limitation of this minimality criteria which in principle results in overfitting. Next, we study and propose a few spatial distribution measures to augment the evaluation criteria. Thereafter, we propose a novel patch-based, white-point centric approach that processes luminance and chrominance information separately to improve on the color matching task. We compare our method qualitatively with several state-of-the art methods using our augmented evaluation criteria along with quantitative examinations. Finally, we perform rigorous experiments and demonstrate results to clearly establish the benefits of our proposed method.
3D Object Tracking for Rough Models
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Song, Xiuqiang; Xie, Weijian; Li, Jiachen; Wang, Nan; Zhong, Fan; Zhang, Guofeng; Qin, Xueying; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Visual monocular 6D pose tracking methods for textureless or weakly-textured objects heavily rely on contour constraints established by the precise 3D model. However, precise models are not always available in reality, and rough models can potentially degrade tracking performance and impede the widespread usage of 3D object tracking. To address this new problem, we propose a novel tracking method that handles rough models. We reshape the rough contour through the probability map, which can avoid explicitly processing the 3D rough model itself. We further emphasize the inner region information of the object, where the points are sampled to provide color constrains. To sufficiently satisfy the assumption of small displacement between frames, the 2D translation of the object is pre-searched for a better initial pose. Finally, we combine constraints from both the contour and inner region to optimize the object pose. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on both roughly and precisely modeled objects. Particularly for the highly rough model, the accuracy is significantly improved (40.4% v.s. 16.9%).
Multi-Level Implicit Function for Detailed Human Reconstruction by Relaxing SMPL Constraints
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Ma, Xikai; Zhao, Jieyu; Teng, Yiqing; Yao, Li; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Aiming at enhancing the rationality and robustness of the results of single-view image-based human reconstruction and acquiring richer surface details, we propose a multi-level reconstruction framework based on implicit functions.This framework first utilizes the predicted SMPL model (Skinned Multi-Person Linear Model) as a prior to further predict consistent 2.5D sketches (depth map and normal map), and then obtains a coarse reconstruction result through an Implicit Function fitting network (IF-Net). Subsequently, with a pixel-aligned feature extraction module and a fine IF-Net, the strong constraints imposed by SMPL are relaxed to add more surface details to the reconstruction result and remove noise. Finally, to address the trade-off between surface details and rationality under complex poses, we propose a novel fusion repair algorithm that reuses existing information. This algorithm compensates for the missing parts of the fine reconstruction results with the coarse reconstruction results, leading to a robust, rational, and richly detailed reconstruction. The final experiments prove the effectiveness of our method and demonstrate that it achieves the richest surface details while ensuring rationality. The project website can be found at https://github.com/MXKKK/2.5D-MLIF.
World-Space Spatiotemporal Path Resampling for Path Tracing
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Zhang, Hangyu; Wang, Beibei; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
With the advent of hardware-accelerated ray tracing, more and more real-time rendering applications tend to render images with ray-traced global illumination (GI). However, the low sample counts at real-time framerates bring enormous challenges to existing path sampling methods. Recent work (ReSTIR GI) samples indirect illumination effectively with a dramatic bias reduction. However, as a screen-space based path resampling approach, it can only reuse the path at the first bounce and brings subtle benefits for complex scenes. To this end, we propose a world-space based spatiotemporal path resampling approach. Our approach caches more path samples into a world-space grid, which allows reusing sub-path starting from non-primary path vertices. Furthermore, we introduce a practical normal-aware hash grid construction approach, providing more efficient candidate samples for path resampling. Eventually, our method achieves improvements ranging from 16.6% to 41.9% in terms of mean squared errors (MSE) compared against the previous method with only 4.4% ~ 8.4% extra time cost.
Enhancing Low-Light Images: A Variation-based Retinex with Modified Bilateral Total Variation and Tensor Sparse Coding
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Yang, Weipeng; Gao, Hongxia; Zou, Wenbin; Huang, Shasha; Chen, Hongsheng; Ma, Jianliang; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Low-light conditions often result in the presence of significant noise and artifacts in captured images, which can be further exacerbated during the image enhancement process, leading to a decrease in visual quality. This paper aims to present an effective low-light image enhancement model based on the variation Retinex model that successfully suppresses noise and artifacts while preserving image details. To achieve this, we propose a modified Bilateral Total Variation to better smooth out fine textures in the illuminance component while maintaining weak structures. Additionally, tensor sparse coding is employed as a regularization term to remove noise and artifacts from the reflectance component. Experimental results on extensive and challenging datasets demonstrate the effectiveness of the proposed method, exhibiting superior or comparable performance compared to state-ofthe- art approaches. Code, dataset and experimental results are available at https://github.com/YangWeipengscut/BTRetinex.
Fabricatable 90° Pop-ups: Interactive Transformation of a 3D Model into a Pop-up Structure
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Fujikawa, Junpei; Ijiri, Takashi; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Ninety-degree pop-ups are a type of papercraft on which a three-dimensional (3D) structure pops up when the angle of the base fold is 90°. They are fabricated by cutting and creasing a single sheet of paper. Traditional 90° pop-ups are limited to 3D shapes only comprising planar shapes because they are made of paper. In this paper, we present novel pop-ups, fabricatable 90° pop-ups that employ the 90° pop-up mechanism, consist of components with curved shapes, and can be fabricatable using a 3D printer. We propose a method for converting a 3D model into a fabricatable 90° pop-up. The user first interactively designs a layout of pop-up components, and the system automatically deforms the components using the 3D model. Because the generated pop-ups contain necessary cuts and folds, no additional assembly process is required. To demonstrate the feasibility of the proposed method, we designed and fabricated various 90° pop-ups using a 3D printer.
Authoring Terrains with Spatialised Style
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Perche, Simon; Peytavie, Adrien; Benes, Bedrich; Galin, Eric; Guérin, Eric; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Various terrain modelling methods have been proposed for the past decades, providing efficient and often interactive authoring tools. However, they seldom include any notion of style, which is critical for designers in the entertainment industry. We introduce a new generative network method that bridges the gap between automatic terrain synthesis and authoring, providing a versatile set of authoring tools allowing spatialised style. We build upon the StyleGAN2 architecture and extend it with authoring tools. Given an input sketch or existing elevation map, our method generates a terrain with features that can be authored, enhanced, and augmented using interactive brushes and style manipulation tools. The strength of our approach lies in the versatility and interoperability of the different tools. We validate our method quantitatively with drainage calculation against other previous techniques and qualitatively by asking users to follow a prompt or freely create a terrain.
Fast Grayscale Morphology for Circular Window
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Moroto, Yuji; Umetani, Nobuyuki; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Morphological operations are among the most popular classic image filters. The filter assumes the maximum or minimum value within a window and is often used for light object thickening and thinning operations, which are important components of various workflows, such as object recognition and stylization. Circular windows are preferred over rectangular windows for obtaining isotropic filter results. However, the existing efficient algorithms focus on rectangular or binary input images. Efficient morphological operations with circular windows for grayscale images remain challenging. In this study, we present a fast grayscale morphology heuristic computation algorithm that decomposes circular windows using the convex hull of circles. We significantly accelerate traditional methods based on Minkowski addition by introducing new decomposition rules specialized for circular windows. As our morphological operation using a convex hull can be computed independently for each pixel, the algorithm is efficient for modern multithreaded hardware.
Efficient Interpolation of Rough Line Drawings
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Chen, Jiazhou; Zhu, Xinding; Even, Melvin; Basset, Jean; Bénard, Pierre; Barla, Pascal; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
In traditional 2D animation, sketches drawn at distant keyframes are used to design motion, yet it would be far too laborintensive to draw all the inbetween frames to fully visualize that motion. We propose a novel efficient interpolation algorithm that generates these intermediate frames in the artist's drawing style. Starting from a set of registered rough vector drawings, we first generate a large number of candidate strokes during a pre-process, and then, at each intermediate frame, we select the subset of those that appropriately conveys the underlying interpolated motion, interpolates the stroke distributions of the key drawings, and introduces a minimum amount of temporal artifacts. In addition, we propose quantitative error metrics to objectively evaluate different stroke selection strategies. We demonstrate the potential of our method on various animations and drawing styles, and show its superiority over competing raster- and vector-based methods.
MAPMaN: Multi-Stage U-Shaped Adaptive Pattern Matching Network for Semantic Segmentation of Remote Sensing Images
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Hong, Tingfeng; Ma, Xiaowen; Wang, Xinyu; Che, Rui; Hu, Chenlu; Feng, Tian; Zhang, Wei; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Remote sensing images (RSIs) often possess obvious background noises, exhibit a multi-scale phenomenon, and are characterized by complex scenes with ground objects in diversely spatial distribution pattern, bringing challenges to the corresponding semantic segmentation. CNN-based methods can hardly address the diverse spatial distributions of ground objects, especially their compositional relationships, while Vision Transformers (ViTs) introduce background noises and have a quadratic time complexity due to dense global matrix multiplications. In this paper, we introduce Adaptive Pattern Matching (APM), a lightweight method for long-range adaptive weight aggregation. Our APM obtains a set of pixels belonging to the same spatial distribution pattern of each pixel, and calculates the adaptive weights according to their compositional relationships. In addition, we design a tiny U-shaped network using the APM as a module to address the large variance of scales of ground objects in RSIs. This network is embedded after each stage in a backbone network to establish a Multi-stage U-shaped Adaptive Pattern Matching Network (MAPMaN), for nested multi-scale modeling of ground objects towards semantic segmentation of RSIs. Experiments on three datasets demonstrate that our MAPMaN can outperform the state-of-the-art methods in common metrics. The code can be available at https://github.com/INiid/MAPMaN.
H-ETC2: Design of a CPU-GPU Hybrid ETC2 Encoder
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Lee, Hyeon-ki; Nah, Jae-Ho; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
This paper proposes a novel CPU-GPU hybrid encoding method based on the ETC2 format, commonly used on mobile platforms. Traditional texture compression techniques often face a trade-off between encoding speed and quality. For a better trade-off, our approach utilizes both the CPU and GPU. In a pipeline we designed, the CPU encoder identifies problematic pixel blocks during the encoding process, and the GPU encoder re-encodes them. Additionally, we carefully improve the base CPU and GPU encoders regarding encoding speed and quality. As a result, our encoder minimizes compression artifacts, increases encoding speed, or achieves both of these goals compared to previous high-quality offline ETC2 encoders.
Controllable Garment Image Synthesis Integrated with Frequency Domain Features
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Liang, Xinru; Mo, Haoran; Gao, Chengying; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Using sketches and textures to synthesize garment images is able to conveniently display the realistic visual effect in the design phase, which greatly increases the efficiency of fashion design. Existing garment image synthesis methods from a sketch and a texture tend to fail in working on complex textures, especially those with periodic patterns. We propose a controllable garment image synthesis framework that takes as inputs an outline sketch and a texture patch and generates garment images with complicated and diverse texture patterns. To improve the performance of global texture expansion, we exploit the frequency domain features in the generative process, which are from a Fast Fourier Transform (FFT) and able to represent the periodic information of the patterns. We also introduce a perceptual loss in the frequency domain to measure the similarity of two texture pattern patches in terms of their intrinsic periodicity and regularity. Comparisons with existing approaches and sufficient ablation studies demonstrate the effectiveness of our method that is capable of synthesizing impressive garment images with diverse texture patterns while guaranteeing proper texture expansion and pattern consistency.
Reconstructing 3D Human Pose from RGB-D Data with Occlusions
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Dang, Bowen; Zhao, Xi; Zhang, Bowen; Wang, He; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
We propose a new method to reconstruct the 3D human body from RGB-D images with occlusions. The foremost challenge is the incompleteness of the RGB-D data due to occlusions between the body and the environment, leading to implausible reconstructions that suffer from severe human-scene penetration. To reconstruct a semantically and physically plausible human body, we propose to reduce the solution space based on scene information and prior knowledge. Our key idea is to constrain the solution space of the human body by considering the occluded body parts and visible body parts separately: modeling all plausible poses where the occluded body parts do not penetrate the scene, and constraining the visible body parts using depth data. Specifically, the first component is realized by a neural network that estimates the candidate region named the "free zone", a region carved out of the open space within which it is safe to search for poses of the invisible body parts without concern for penetration. The second component constrains the visible body parts using the "truncated shadow volume" of the scanned body point cloud. Furthermore, we propose to use a volume matching strategy, which yields better performance than surface matching, to match the human body with the confined region. We conducted experiments on the PROX dataset, and the results demonstrate that our method produces more accurate and plausible results compared with other methods.
CP-NeRF: Conditionally Parameterized Neural Radiance Fields for Cross-scene Novel View Synthesis
(The Eurographics Association and John Wiley & Sons Ltd., 2023) He, Hao; Liang, Yixun; Xiao, Shishi; Chen, Jierun; Chen, Yingcong; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Neural radiance fields (NeRF) have demonstrated a promising research direction for novel view synthesis. However, the existing approaches either require per-scene optimization that takes significant computation time or condition on local features which overlook the global context of images. To tackle this shortcoming, we propose the Conditionally Parameterized Neural Radiance Fields (CP-NeRF), a plug-in module that enables NeRF to leverage contextual information from different scales. Instead of optimizing the model parameters of NeRFs directly, we train a Feature Pyramid hyperNetwork (FPN) that extracts view-dependent global and local information from images within or across scenes to produce the model parameters. Our model can be trained end-to-end with standard photometric loss from NeRF. Extensive experiments demonstrate that our method can significantly boost the performance of NeRF, achieving state-of-the-art results in various benchmark datasets.
GA-Sketching: Shape Modeling from Multi-View Sketching with Geometry-Aligned Deep Implicit Functions
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Zhou, Jie; Luo, Zhongjin; Yu, Qian; Han, Xiaoguang; Fu, Hongbo; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Sketch-based shape modeling aims to bridge the gap between 2D drawing and 3D modeling by providing an intuitive and accessible approach to create 3D shapes from 2D sketches. However, existing methods still suffer from limitations in reconstruction quality and multi-view interaction friendliness, hindering their practical application. This paper proposes a faithful and user-friendly iterative solution to tackle these limitations by learning geometry-aligned deep implicit functions from one or multiple sketches. Our method lifts 2D sketches to volume-based feature tensors, which align strongly with the output 3D shape, enabling accurate reconstruction and faithful editing. Such a geometry-aligned feature encoding technique is well-suited to iterative modeling since features from different viewpoints can be easily memorized or aggregated. Based on these advantages, we design a unified interactive system for sketch-based shape modeling. It enables users to generate the desired geometry iteratively by drawing sketches from any number of viewpoints. In addition, it allows users to edit the generated surface by making a few local modifications. We demonstrate the effectiveness and practicality of our method with extensive experiments and user studies, where we found that our method outperformed existing methods in terms of accuracy, efficiency, and user satisfaction. The source code of this project is available at https://github.com/LordLiang/GA-Sketching.
Fine Back Surfaces Oriented Human Reconstruction for Single RGB-D Images
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Fang, Xianyong; Qian, Yu; He, Jinshen; Wang, Linbo; Liu, Zhengyi; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Current single RGB-D image based human surface reconstruction methods generally take both the RGB images and the captured frontal depth maps together so that the 3D cues from the frontal surfaces can help infer the full surface geometries. However, we observe that the back surfaces can often be quite different from the frontal surfaces and, therefore, current methods can mess the recovery process by adopting such 3D cues, especially for the unseen back surfaces. We need to do the back surface inference without the frontal depth map. Consequently, a novel human reconstruction framework is proposed, so that human models with fine geometric details, especially for the back surfaces, can be obtained. In this approach, a progressive estimation method is introduced to effectively recover the unseen back depth maps. The coarse back depth maps are recovered by the parametric models of the subjects, with the fine ones further obtained by the normal-maps conditioned GAN. This framework also includes a cross-attention based denoising method for the frontal depth maps. This method adopts the cross attention between the features of the last two layers encoded from the frontal depth maps and thus suppresses the noise for fine depth maps by the attentions of features from the low-noise and globally-structured highest layer. Experimental results show the efficacies of the proposed ideas.
Neural Shading Fields for Efficient Facial Inverse Rendering
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Rainer, Gilles; Bridgeman, Lewis; Ghosh, Abhijeet; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Given a set of unstructured photographs of a subject under unknown lighting, 3D geometry reconstruction is relatively easy, but reflectance estimation remains a challenge. This is because it requires disentangling lighting from reflectance in the ambiguous observations. Solutions exist leveraging statistical, data-driven priors to output plausible reflectance maps even in the underconstrained single-view, unknown lighting setting. We propose a very low-cost inverse optimization method that does not rely on data-driven priors, to obtain high-quality diffuse and specular, albedo and normal maps in the setting of multi-view unknown lighting. We introduce compact neural networks that learn the shading of a given scene by efficiently finding correlations in the appearance across the face. We jointly optimize the implicit global illumination of the scene in the networks with explicit diffuse and specular reflectance maps that can subsequently be used for physically-based rendering. We analyze the veracity of results on ground truth data, and demonstrate that our reflectance maps maintain more detail and greater personal identity than state-of-the-art deep learning and differentiable rendering methods.
Data-Driven Ink Painting Brushstroke Rendering
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Madono, Koki; Simo-Serra, Edgar; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Although digital painting has advanced much in recent years, there is still a significant divide between physically drawn paintings and purely digitally drawn paintings. These differences arise due to the physical interactions between the brush, ink, and paper, which are hard to emulate in the digital domain. Most ink painting approaches have focused on either using heuristics or physical simulation to attempt to bridge the gap between digital and analog, however, these approaches are still unable to capture the diversity of painting effects, such as ink fading or blotting, found in the real world. In this work, we propose a data-driven approach to generate ink paintings based on a semi-automatically collected high-quality real-world ink painting dataset. We use a multi-camera robot-based setup to automatically create a diversity of ink paintings, which allows for capturing the entire process in high resolution, including capturing detailed brush motions and drawing results. To ensure high-quality capture of the painting process, we calibrate the setup and perform occlusion-aware blending to capture all the strokes in high resolution in a robust and efficient way. Using our new dataset, we propose a recursive deep learning-based model to reproduce the ink paintings stroke by stroke while capturing complex ink painting effects such as bleeding and mixing. Our results corroborate the fidelity of the proposed approach to real hand-drawn ink paintings in comparison with existing approaches. We hope the availability of our dataset will encourage new research on digital realistic ink painting techniques.
Interactive Authoring of Terrain using Diffusion Models
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Lochner, Joshua; Gain, James; Perche, Simon; Peytavie, Adrien; Galin, Eric; Guérin, Eric; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Generating heightfield terrains is a necessary precursor to the depiction of computer-generated natural scenes in a variety of applications. Authoring such terrains is made challenging by the need for interactive feedback, effective user control, and perceptually realistic output encompassing a range of landforms.We address these challenges by developing a terrain-authoring framework underpinned by an adaptation of diffusion models for conditional image synthesis, trained on real-world elevation data. This framework supports automated cleaning of the training set; authoring control through style selection and feature sketches; the ability to import and freely edit pre-existing terrains, and resolution amplification up to the limits of the source data. Our framework improves on previous machine-learning approaches by: expanding landform variety beyond mountainous terrain to encompass cliffs, canyons, and plains; providing a better balance between terseness and specificity in user control, and improving the fidelity of global terrain structure and perceptual realism. This is demonstrated through drainage simulations and a user study testing the perceived realism for different classes of terrain. The full source code, blender add-on, and pretrained models are available.

Browse

Browsing 42-Issue 7 by Issue Date

Results Per Page

Sort Options