39-Issue 7

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/2632946

Browse

Now showing 1 - 20 of 54

Next Event Estimation++: Visibility Mapping for Efficient Light Transport Simulation
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Guo, Jerry Jinfeng; Eisemann, Martin; Eisemann, Elmar; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Monte-Carlo rendering requires determining the visibility between scene points as the most common and compute intense operation to establish paths between camera and light source. Unfortunately, many tests reveal occlusions and the corresponding paths do not contribute to the final image. In this work, we present next event estimation++ (NEE++): a visibility mapping technique to perform visibility tests in a more informed way by caching voxel to voxel visibility probabilities. We show two scenarios: Russian roulette style rejection of visibility tests and direct importance sampling of the visibility. We show applications to next event estimation and light sampling in a uni-directional path tracer, and light-subpath sampling in Bi-Directional Path Tracing. The technique is simple to implement, easy to add to existing rendering systems, and comes at almost no cost, as the required information can be directly extracted from the rendering process itself. It discards up to 80% of visibility tests on average, while reducing variance by ~20% compared to other state-of-the-art light sampling techniques with the same number of samples. It gracefully handles complex scenes with efficiency similar to Metropolis light transport techniques but with a more uniform convergence.
Cosserat Rod with rh-Adaptive Discretization
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Wen, Jiahao; Chen, Jiong; Nobuyuki, Umetani; Bao, Hujun; Huang, Jin; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Rod-like one-dimensional elastic objects often exhibit complex behaviors which pose great challenges to the discretization method for pursuing a faithful simulation. By only moving a small portion of material points, the Eulerian-on-Lagrangian (EoL) method already shows great adaptivity to handle sharp contact, but it is still far from enough to reproduce rich and complex geometry details arising in simulations. In this paper, we extend the discrete configuration space by unifying all Lagrangian and EoL nodes in representation for even more adaptivity with every sample being assigned with a dynamic material coordinate. However, this great extension will immediately bring in much more redundancy in the dynamic system. Therefore, we propose additional energy to control the spatial distribution of all material points, seeking to equally space them with respect to a curvature-based density field as a monitor. This flexible approach can effectively constrain the motion of material points to resolve numerical degeneracy, while simultaneously enables them to notably slide inside the parametric domain to account for the shape parameterization. Besides, to accurately respond to sharp contact, our method can also insert or remove nodes online and adjust the energy stiffness to suppress possible jittering artifacts that could be excited in a stiff system. As a result of this hybrid rh-adaption, our proposed method is capable of reproducing many realistic rod dynamics, such as excessive bending, twisting and knotting while only using a limited number of elements.
Practical Fabrication of Discrete Chebyshev Nets
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Liu, Hao-Yu; Liu, Zhong-Yuan; Zhao, Zheng-Yu; Liu, Ligang; Fu, Xiao-Ming; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
We propose a computational and practical technique to allow home users to fabricate discrete Chebyshev nets for various 3D models. The success of our method relies on two key components. The first one is a novel and simple method to approximate discrete integrable, unit-length, and angle-bounded frame fields, used to model discrete Chebyshev nets. Central to our field generation process is an alternating algorithm that takes turns executing one pass to enforce integrability and another pass to approach unit length while bounding angles. The second is a practical fabrication specification. The discrete Chebyshev net is first partitioned into a set of patches to facilitate manufacturing. Then, each patch is assigned a specification on pulling, bend, and fold to fit the nets. We demonstrate the capability and feasibility of our method in various complex models.
A Multi-Person Selfie System via Augmented Reality
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Lin, Jie; Yang, Chuan-Kai; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
The limited length of a selfie stick always poses the problem of distortion in a selfie, in spite of the prevalence of selfie stick in recent years.We propose a technique, based on modifying existing augmented reality technology, to support the selfie of multiple persons, through properly aligning different photographing processes. It can be shown that our technique helps avoiding the common distortion drawback of using a selfie stick, and facilitates the composition process of a group photo. It can also be used to create some special effects, including creating an illusion of having multiple appearances of a person.
Personalized Hand Modeling from Multiple Postures with Multi-View Color Images
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Wang, Yangang; Rao, Ruting; Zou, Changqing; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Personalized hand models can be utilized to synthesize high quality hand datasets, provide more possible training data for deep learning and improve the accuracy of hand pose estimation. In recent years, parameterized hand models, e.g., MANO, are widely used for obtaining personalized hand models. However, due to the low resolution of existing parameterized hand models, it is still hard to obtain high-fidelity personalized hand models. In this paper, we propose a new method to estimate personalized hand models from multiple hand postures with multi-view color images. The personalized hand model is represented by a personalized neutral hand, and multiple hand postures. We propose a novel optimization strategy to estimate the neutral hand from multiple hand postures. To demonstrate the performance of our method, we have built a multi-view system and captured more than 35 people, and each of them has 30 hand postures.We hope the estimated hand models can boost the research of highfidelity parameterized hand modeling in the future. All the hand models are publicly available on www.yangangwang.com.
Multi-scale Information Assembly for Image Matting
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Qiao, Yu; Liu, Yuhao; Zhu, Qiang; Yang, Xin; Wang, Yuxin; Zhang, Qiang; Wei, Xiaopeng; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Image matting is a long-standing problem in computer graphics and vision, mostly identified as the accurate estimation of the foreground in input images.We argue that the foreground objects can be represented by different-level information, including the central bodies, large-grained boundaries, refined details, etc. Based on this observation, in this paper, we propose a multi-scale information assembly framework (MSIA-matte) to pull out high-quality alpha mattes from single RGB images. Technically speaking, given an input image, we extract advanced semantics as our subject content and retain initial CNN features to encode different-level foreground expression, then combine them by our well-designed information assembly strategy. Extensive experiments can prove the effectiveness of the proposed MSIA-matte, and we can achieve state-of-the-art performance compared to most existing matting networks.
Automatic Band-Limited Approximation of Shaders Using Mean-Variance Statistics in Clamped Domain
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Li, Shi; Wang, Rui; Huo, Yuchi; Zheng, Wenting; Hua, Wei; Bao, Hujun; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
In this paper, we present a new shader smoothing method to improve the quality and generality of band-limiting shader programs. Previous work [YB18] treats intermediate values in the program as random variables, and utilizes mean and variance statistics to smooth shader programs. In this work, we extend such a band-limiting framework by exploring the observation that one intermediate value in the program is usually computed by a complex composition of functions, where the domain and range of composited functions heavily impact the statistics of smoothed programs. Accordingly, we propose three new shader smoothing rules for specific composition of functions by considering the domain and range, enabling better mean and variance statistics of approximations. Aside from continuous functions, the texture, such as color texture or normal map, is treated as a discrete function with limited domain and range, thereby can be processed similarly in the newly proposed framework. Experiments show that compared with previous work, our method is capable of generating better smoothness of shader programs as well as handling a broader set of shader programs.
Weakly Supervised Part-wise 3D Shape Reconstruction from Single-View RGB Images
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Niu, Chengjie; Yu, Yang; Bian, Zhenwei; Li, Jun; Xu, Kai; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
In order for the deep learning models to truly understand the 2D images for 3D geometry recovery, we argue that singleview reconstruction should be learned in a part-aware and weakly supervised manner. Such models lead to more profound interpretation of 2D images in which part-based parsing and assembling are involved. To this end, we learn a deep neural network which takes a single-view RGB image as input, and outputs a 3D shape in parts represented by 3D point clouds with an array of 3D part generators. In particular, we devise two levels of generative adversarial network (GAN) to generate shapes with both correct part shape and reasonable overall structure. To enable a self-taught network training, we devise a differentiable projection module along with a self-projection loss measuring the error between the shape projection and the input image. The training data in our method is unpaired between the 2D images and the 3D shapes with part decomposition. Through qualitative and quantitative evaluations on public datasets, we show that our method achieves good performance in part-wise single-view reconstruction.
Pixel-wise Dense Detector for Image Inpainting
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Zhang, Ruisong; Quan, Weize; Wu, Baoyuan; Li, Zhifeng; Yan, Dong-Ming; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Recent GAN-based image inpainting approaches adopt an average strategy to discriminate the generated image and output a scalar, which inevitably lose the position information of visual artifacts. Moreover, the adversarial loss and reconstruction loss (e.g., `1 loss) are combined with tradeoff weights, which are also difficult to tune. In this paper, we propose a novel detection-based generative framework for image inpainting, which adopts the min-max strategy in an adversarial process. The generator follows an encoder-decoder architecture to fill the missing regions, and the detector using weakly supervised learning localizes the position of artifacts in a pixel-wise manner. Such position information makes the generator pay attention to artifacts and further enhance them. More importantly, we explicitly insert the output of the detector into the reconstruction loss with a weighting criterion, which balances the weight of the adversarial loss and reconstruction loss automatically rather than manual operation. Experiments on multiple public datasets show the superior performance of the proposed framework. The source code is available at https://github.com/Evergrow/GDN_Inpainting.
The Layerizing VoxPoint Annular Convolutional Network for 3D Shape Classification
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Wang, Tong; Tao, Wenyuan; Own, Chung-Ming; Lou, Xiantuo; Zhao, Yuehua; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Analyzing the geometric and semantic properties of 3D point cloud data via the deep learning networks is still challenging due to the irregularity and sparsity of samplings of their geometric structures. In our study, the authors combine the advantage of voxels and point clouds by presenting a new data form of voxel models, called Layer-Ring data. This data type can retain the fine description of the 3D data, and keep the high efficiency of feature extraction. After that, based on the Layer-Ring data, a modern network architecture, called VoxPoint Annular Network (VAN), works on the Layer-Ring data for the feature extraction and object category prediction. The design idea is based on the edge-extraction and the coordinate representation for each voxel on the separated layer. With the flexible design, our proposed VAN can adapt to the layer's geometric variability and scalability. Finally, the extensive experiments and comparisons demonstrate that our approach obtained the notable results with the state-of-the-art methods on a variety of standard benchmark datasets (e.g., ModelNet10, ModelNet40). Moreover, the tests also proved that 3D shape features could learn efficiently and robustly. All relevant codes will be available at https://github.com/helloFionaQ/Vox-PointNet.
Colorization of Line Drawings with Empty Pupils
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Akita, Kenta; Morimoto, Yuki; Tsuruno, Reiji; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Many studies have recently applied deep learning to the automatic colorization of line drawings. However, it is difficult to paint empty pupils using existing methods because the convolutional neural network are trained with pupils that have edges, which are generated from color images using image processing. Most actual line drawings have empty pupils that artists must paint in. In this paper, we propose a novel network model that transfers the pupil details in a reference color image to input line drawings with empty pupils. We also propose a method for accurately and automatically colorizing eyes. In this method, eye patches are extracted from a reference color image and automatically added to an input line drawing as color hints using our pupil position estimation network.
SRF-Net: Spatial Relationship Feature Network for Tooth Point Cloud Classification
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Ma, Qian; Wei, Guangshun; Zhou, Yuanfeng; Pan, Xiao; Xin, Shiqing; Wang, Wenping; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
3D scanned point cloud data of teeth is popular used in digital orthodontics. The classification and semantic labelling for point cloud of each tooth is a key and challenging task for planning dental treatment. Utilizing the priori ordered position information of tooth arrangement, we propose an effective network for tooth model classification in this paper. The relative position and the adjacency similarity feature vectors are calculated for tooth 3D model, and combine the geometric feature into the fully connected layers of the classification training task. For the classification of dental anomalies, we present a dental anomalies processing method to improve the classification accuracy. We also use FocalLoss as the loss function to solve the sample imbalance of wisdom teeth. The extensive evaluations, ablation studies and comparisons demonstrate that the proposed network can classify tooth models accurately and automatically and outperforms state-of-the-art point cloud classification methods.
Unsupervised Image Reconstruction for Gradient-Domain Volumetric Rendering
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Xu, Zilin; Sun, Qiang; Wang, Lu; Xu, Yanning; Wang, Beibei; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Gradient-domain rendering can highly improve the convergence of light transport simulation using the smoothness in image space. These methods generate image gradients and solve an image reconstruction problem with rendered image and the gradient images. Recently, a previous work proposed a gradient-domain volumetric photon density estimation for homogeneous participating media. However, the image reconstruction relies on traditional L1 reconstruction, which leads to obvious artifacts when only a few rendering passes are performed. Deep learning based reconstruction methods have been exploited for surface rendering, but they are not suitable for volume density estimation. In this paper, we propose an unsupervised neural network for image reconstruction of gradient-domain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet with an encoded shift connection and a separated auxiliary feature branch, which includes volume based auxiliary features such as transmittance and photon density. Our network smooths the images on global scale and preserves the high frequency details on a small scale. We demonstrate that our network produces a higher quality result, compared to previous work. Although we only considered volumetric photon mapping, it's straightforward to extend our method for other forms, like beam radiance estimation.
A Novel Plastic Phase-Field Method for Ductile Fracture with GPU Optimization
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Zhao, Zipeng; Huang, Kemeng; Li, Chen; Wang, Changbo; Qin, Hong; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
efficiently simulate ductile fracture with GPU optimization. At the theoretical level of physically-based modeling and simulation, our PPF approach assumes the fracture sensitivity of the material increases with the plastic strain accumulation. As a result, we first develop a hardening-related fracture toughness function towards phase-field evolution. Second, we follow the associative flow rule and adopt a novel degraded von Mises yield criterion. In this way, we establish the tight coupling of the phase-field and plastic treatment, with which our PPF method can present distinct elastoplasticity, necking, and fracture characteristics during ductile fracture simulation. At the numerical level towards GPU optimization, we further devise an advanced parallel framework, which takes the full advantages of hierarchical architecture. Our strategy dramatically enhances the computational efficiency of preprocessing and phase-field evolution for our PPF with the material point method (MPM). Based on our extensive experiments on a variety of benchmarks, our novel method's performance gain can reach 1.56x speedup of the primary GPU MPM. Finally, our comprehensive simulation results have confirmed that this new PPF method can efficiently and realistically simulate complex ductile fracture phenomena in 3D interactive graphics and animation.
InstanceFusion: Real-time Instance-level 3D Reconstruction Using a Single RGBD Camera
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Lu, Feixiang; Peng, Haotian; Wu, Hongyu; Yang, Jun; Yang, Xinhang; Cao, Ruizhi; Zhang, Liangjun; Yang, Ruigang; Zhou, Bin; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
We present InstanceFusion, a robust real-time system to detect, segment, and reconstruct instance-level 3D objects of indoor scenes with a hand-held RGBD camera. It combines the strengths of deep learning and traditional SLAM techniques to produce visually compelling 3D semantic models. The key success comes from our novel segmentation scheme and the efficient instancelevel data fusion, which are both implemented on GPU. Specifically, for each incoming RGBD frame, we take the advantages of the RGBD features, the 3D point cloud, and the reconstructed model to perform instance-level segmentation. The corresponding RGBD data along with the instance ID are then fused to the surfel-based models. In order to sufficiently store and update these data, we design and implement a new data structure using the OpenGL Shading Language. Experimental results show that our method advances the state-of-the-art (SOTA) methods in instance segmentation and data fusion by a big margin. In addition, our instance segmentation improves the precision of 3D reconstruction, especially in the loop closure. InstanceFusion system runs 20.5Hz on a consumer-level GPU, which supports a number of augmented reality (AR) applications (e.g., 3D model registration, virtual interaction, AR map) and robot applications (e.g., navigation, manipulation, grasping). To facilitate future research and reproduce our system more easily, the source code, data, and the trained model are released on Github: https://github.com/Fancomi2017/InstanceFusion.
Image-Driven Furniture Style for Interactive 3D Scene Modeling
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Weiss, Tomer; Yildiz, Ilkay; Agarwal, Nitin; Ataer-Cansizoglu, Esra; Choi, Jae-Woo; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Creating realistic styled spaces is a complex task, which involves design know-how for what furniture pieces go well together. Interior style follows abstract rules involving color, geometry and other visual elements. Following such rules, users manually select similar-style items from large repositories of 3D furniture models, a process which is both laborious and time-consuming. We propose a method for fast-tracking style-similarity tasks, by learning a furniture's style-compatibility from interior scene images. Such images contain more style information than images depicting single furniture. To understand style, we train a deep learning network on a classification task. Based on image embeddings extracted from our network, we measure stylistic compatibility of furniture. We demonstrate our method with several 3D model style-compatibility results, and with an interactive system for modeling style-consistent scenes.
Generating High-quality Superpixels in Textured Images
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Zhang, Zhe; Xu, Panpan; Chang, Jian; Wang, Wencheng; Zhao, Chong; Zhang, Jian Jun; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Superpixel segmentation is important for promoting various image processing tasks. However, existing methods still have difficulties in generating high-quality superpixels in textured images, because they cannot separate textures from structures well. Though texture filtering can be adopted for smoothing textures before superpixel segmentation, the filtering would also smooth the object boundaries, and thus weaken the quality of generated superpixels. In this paper, we propose to use the adaptive scale box smoothing instead of the texture filtering to obtain more high-quality texture and boundary information. Based on this, we design a novel distance metric to measure the distance between different pixels, which considers boundary, color and Euclidean distance simultaneously. As a result, our method can achieve high-quality superpixel segmentation in textured images without texture filtering. The experimental results demonstrate the superiority of our method over existing methods, even the learning-based methods. Benefited from using boundaries to guide superpixel segmentation, our method can also suppress noise to generate high-quality superpixels in non-textured images.
A Graph-based One-Shot Learning Method for Point Cloud Recognition
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Fan, Zhaoxin; Liu, Hongyan; He, Jun; Sun, Qi; Du, Xiaoyong; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Point cloud based 3D vision tasks, such as 3D object recognition, are critical to many real world applications such as autonomous driving. Many point cloud processing models based on deep learning have been proposed by researchers recently. However, they are all large-sample dependent, which means that a large amount of manually labelled training data are needed to train the model, resulting in huge labor cost. In this paper, to tackle this problem, we propose a One-Shot learning model for Point Cloud Recognition, namely OS-PCR. Different from previous methods, our method formulates a new setting, where the model only needs to see one sample per class once for memorizing at inference time when new classes are needed to be recognized. To fulfill this task, we design three modules in the model: an Encoder Module, an Edge-conditioned Graph Convolutional Network Module, and a Query Module. To evaluate the performance of the proposed model, we build a one-shot learning benchmark dataset for 3D point cloud analysis. Then, comprehensive experiments are conducted on it to demonstrate the effectiveness of our proposed model.
CLA-GAN: A Context and Lightness Aware Generative Adversarial Network for Shadow Removal
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Zhang, Ling; Long, Chengjiang; Yan, Qingan; Zhang, Xiaolong; Xiao, Chunxia; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
In this paper, we propose a novel context and lightness aware Generative Adversarial Network (CLA-GAN) framework for shadow removal, which refines a coarse result to a final shadow removal result in a coarse-to-fine fashion. At the refinement stage, we first obtain a lightness map using an encoder-decoder structure. With the lightness map and the coarse result as the inputs, the following encoder-decoder tries to refine the final result. Specifically, different from current methods restricted pixel-based features from shadow images, we embed a context-aware module into the refinement stage, which exploits patch-based features. The embedded module transfers features from non-shadow regions to shadow regions to ensure the consistency in appearance in the recovered shadow-free images. Since we consider pathces, the module can additionally enhance the spatial association and continuity around neighboring pixels. To make the model pay more attention to shadow regions during training, we use dynamic weights in the loss function. Moreover, we augment the inputs of the discriminator by rotating images in different degrees and use rotation adversarial loss during training, which can make the discriminator more stable and robust. Extensive experiments demonstrate the validity of the components in our CLA-GAN framework. Quantitative evaluation on different shadow datasets clearly shows the advantages of our CLA-GAN over the state-of-the-art methods.
Adjustable Constrained Soft-Tissue Dynamics
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Wang, Bohan; Zheng, Mianlun; Barbic, Jernej; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Physically based simulation is often combined with geometric mesh animation to add realistic soft-body dynamics to virtual characters. This is commonly done using constraint-based simulation whereby a soft-tissue simulation is constrained to geometric animation of a subpart (or otherwise proxy representation) of the character. We observe that standard constraint-based simulation suffers from an important flaw that limits the expressiveness of soft-body dynamics. Namely, under correct physics, the frequency and amplitude of soft-tissue dynamics arising from constraints (''inertial amplitude'') are coupled, and cannot be adjusted independently merely by adjusting the material properties of the model. This means that the space of physically based simulations is inherently limited and cannot capture all effects typically expected by computer animators. For example, animators need the ability to adjust the frequency, inertial amplitude, gravity sag and damping properties of the virtual character, independently from each other, as these are the primary visual characteristics of the soft-tissue dynamics. We demonstrate that independence can be achieved by transforming the equations of motion into a non-inertial reference coordinate frame, then scaling the resulting inertial forces, and then converting the equations of motion back to the inertial frame. Such scaling of inertia makes it possible for the animator to set the character's inertial amplitude independently from frequency. We also provide exact controls for the amount of character's gravity sag, and the damping properties. In our examples, we use linear blend skinning and pose-space deformation for geometric mesh animation, and the Finite Element Method for soft-body constrained simulation; but our idea of scaling inertial forces is general and applicable to other animation and simulation methods. We demonstrate our technique on several character examples.

Browse

Browsing 39-Issue 7 by Issue Date

Results Per Page

Sort Options