PG: Pacific Graphics Short Papers

Permanent URI for this community

https://diglib.eg.org/handle/10.2312/995

Browse

Now showing 1 - 20 of 265

3D Human Body Skeleton Extraction from Consecutive Surfaces
(The Eurographics Association, 2019) Zhang, Yong; Tan, Fei; Wang, Shaofan; Kong, Dehui; Yin, Baocai; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
Extracting human body skeletons from consecutive surfaces is an important research topic in the fields of computer graphics and human computer interaction, especially in posture estimation and skeleton animation. Current approaches mainly suffer from following problems: insufficient time and space continuity, not robust to background, ambient noise, etc. Our approach is to improve against these shortcomings. This paper proposes a 3D human body skeleton extraction method from consecutive meshes. We extract the consistent skeletons from consecutive surfaces based on shape segmentation and construct skeleton sequences, then we use the continuous frame skeleton point optimization model we proposed to optimize the skeleton sequences, generating the final skeleton point sequences which are more accurate. Finally, we verify that our method can obtain more complete and accurate skeletons compared to other methods through many experiments.
3D VAE-Attention Network: A Parallel System for Single-view 3D Reconstruction
(The Eurographics Association, 2018) Hu, Fei; Yang, Xinyan; Zhong, Wei; Ye, Long; Zhang, Qin; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
3D object reconstruction from single view image is a challenge task. Due to the fact that the information contained in one isolated image is not sufficient for reasonable 3D shape reconstruction, the existing results on single-view 3D reconstruction always lack marginal voxels. To tackle this problem, we propose a parallel system named 3D VAE-attention network (3VAN) for single view 3D reconstruction. Distinct from the common encoder-decoder structure, the proposed network consists of two parallel branches, 3D-VAE and Attention Network. 3D-VAE completes the general shape reconstruction by an extension of standard VAE model, and Attention Network supplements the missing details by a 3D reconstruction attention network. In the experiments, we verify the feasibility of our 3VAN on the ShapeNet and PASCAL 3D+ datasets. By comparing with the state-of-art methods, the proposed 3VAN can produce more precise 3D object models in terms of both qualitative and quantitative evaluation.
3D-CariNet: End-to-end 3D Caricature Generation from Natural Face Images with Differentiable Renderer
(The Eurographics Association, 2021) Huang, Meijia; Dai, Ju; Pan, Junjun; Bai, Junxuan; Qin, Hong; Lee, Sung-Hee and Zollmann, Stefanie and Okabe, Makoto and Wünsche, Burkhard
Caricatures are an artistic representation of human faces to express satire and humor. Caricature generation of human faces is a hotspot in CG research. Previous work mainly focuses on 2D caricatures generation from face photos or 3D caricature reconstruction from caricature images. In this paper, we propose a novel end-to-end method to directly generate personalized 3D caricatures from a single natural face image. It can create not only exaggerated geometric shapes, but also heterogeneous texture styles. Firstly, we construct a synthetic dataset containing matched data pairs composed of face photos, caricature images, and 3D caricatures. Then, we design a graph convolutional autoencoder to build a non-linear colored mesh model to learn the shape and texture of 3D caricatures. To make the network end-to-end trainable, we incorporate a differentiable renderer to render 3D caricatures into caricature images inversely. Experiments demonstrate that our method can achieve 3D caricature generation with various texture styles from face images while maintaining personality characteristics.
3D-SSGAN: Lifting 2D Semantics for 3D-Aware Compositional Portrait Synthesis
(The Eurographics Association, 2024) Liu, Ruiqi; Zheng, Peng; Wang, Ye; Ma, Rui; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Existing 3D-aware portrait synthesis methods can generate impressive high-quality images while preserving strong 3D consistency. However, most of them cannot support the fine-grained part-level control over synthesized images. Conversely, some GAN-based 2D portrait synthesis methods can achieve clear disentanglement of facial regions, but they cannot preserve view consistency due to a lack of 3D modeling abilities. To address these issues, we propose 3D-SSGAN, a novel framework for 3D-aware compositional portrait image synthesis. First, a simple yet effective depth-guided 2D-to-3D lifting module maps the generated 2D part features and semantics to 3D. Then, a volume renderer with a novel 3D-aware semantic mask renderer is utilized to produce the composed face features and corresponding masks. The whole framework is trained end-to-end by discriminating between real and synthesized 2D images and their semantic masks. Quantitative and qualitative evaluations demonstrate the superiority of 3D-SSGAN in controllable part-level synthesis while preserving 3D view consistency.
3DStyleGLIP: Part-Tailored Text-Guided 3D Neural Stylization
(The Eurographics Association, 2024) Chung, SeungJeh; Park, JooHyun; Kang, HyeongYeop; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
3D stylization, the application of specific styles to three-dimensional objects, offers substantial commercial potential by enabling the creation of uniquely styled 3D objects tailored to diverse scenes. Recent advancements in artificial intelligence and textdriven manipulation methods have made the stylization process increasingly intuitive and automated. While these methods reduce human costs by minimizing reliance on manual labor and expertise, they predominantly focus on holistic stylization, neglecting the application of desired styles to individual components of a 3D object. This limitation restricts the fine-grained controllability. To address this gap, we introduce 3DStyleGLIP, a novel framework specifically designed for text-driven, parttailored 3D stylization. Given a 3D mesh and a text prompt, 3DStyleGLIP utilizes the vision-language embedding space of the Grounded Language-Image Pre-training (GLIP) model to localize individual parts of the 3D mesh and modify their appearance to match the styles specified in the text prompt. 3DStyleGLIP effectively integrates part localization and stylization guidance within GLIP's shared embedding space through an end-to-end process, enabled by part-level style loss and two complementary learning techniques. This neural methodology meets the user's need for fine-grained style editing and delivers high-quality part-specific stylization results, opening new possibilities for customization and flexibility in 3D content creation. Our code and results are available at https://github.com/sj978/3DStyleGLIP.
Accelerating Graph-based Path Planning Through Waypoint Clustering
(The Eurographics Association, 2015) Wardhana, Nicholas Mario; Johan, Henry; Seah, Hock-Soon; Stam, Jos and Mitra, Niloy J. and Xu, Kun
Modern Computer Graphics applications commonly feature very large virtual environments and diverse characters which perform different kinds of motions. To accelerate path planning in such scenario, we propose subregion graph data structure. It consists of subregions, which are clusters of locally connected waypoints inside a region, as well as their connectivities. We also present a fast algorithm to automatically generate subregion graph from enhanced waypoint graph map representation, which also supports various motion types and can be created from large virtual environments. Nevertheless, subregion graph can also be generated from any graph-based map representation. Our experiments showed that subregion graph is very compact relative to the input waypoint graph. By firstly planning subregion path, and then limiting waypoint-level planning to the subregion path, up to 8 times average speedup can be achieved, while average length ratios are maintained at as low as 102.5%.
Adaptive and Dynamic Regularization for Rolling Guidance Image Filtering
(The Eurographics Association, 2022) Fukatsu, Miku; Yoshizawa, Shin; Takemura, Hiroshi; Yokota, Hideo; Yang, Yin; Parakkat, Amal D.; Deng, Bailin; Noh, Seung-Tak
Separating shapes and textures of digital images at different scales is useful in computer graphics. The Rolling Guidance (RG) filter, which removes structures smaller than a specified scale while preserving salient edges, has attracted considerable attention. Conventional RG-based filters have some drawbacks, including smoothness/sharpness quality dependence on scale and non-uniform convergence. This paper proposes a novel RG-based image filter that has more stable filtering quality at varying scales. Our filtering approach is an adaptive and dynamic regularization for a recursive regression model in the RG framework to produce more edge saliency and appropriate scale convergence. Our numerical experiments demonstrated filtering results with uniform convergence and high accuracy for varying scales.
Adaptive Hierarchical Shape Matching
(The Eurographics Association, 2015) Tian, Yuan; Yang, Yin; Guo, Xiaohu; Prabhakaran, Balakrishnan; Stam, Jos and Mitra, Niloy J. and Xu, Kun
In this paper, we present an adaptive hierarchical method allowing users to interact with geometrically complex 3D deformable objects based on an extended shape matching approach. Our method extends the existing multiresolution shape matching methods with improved energy convergence rate. This is achieved by using adaptive integration strategies to avoid insignificant shape matching iterations during the simulation. As demonstrated in our experimental results, the proposed method provides an efficient yet stable deformable simulation of complex models in real-time.
Adaptive Measurement of Anisotropic Material Appearance
(The Eurographics Association, 2017) Vávra, Radomir; Filip, Jiri; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
We present a practical adaptive method for acquisition of the anisotropic BRDF. It is based on a sparse adaptive measurement of the complete four-dimensional BRDF space by means of one-dimensional slices which form a sparse four-dimensional structure in the BRDF space and which can be measured by continuous movements of a light source and a sensor. Such a sampling approach is advantageous especially for gonioreflectometer-based measurement devices where the mechanical travel of a light source and a sensor creates a significant time constraint. In order to evaluate our method, we perform adaptive measurements of three materials and we simulate adaptive measurements of ten others. We achieve a four-times lower reconstruction error in comparison with the regular non-adaptive BRDF measurements given the same count of measured samples. Our method is almost twice better than a previous adaptive method, and it requires from two- to five-times less samples to achieve the same results as alternative approaches.
Aesthetic Enhancement via Color Area and Location Awareness
(The Eurographics Association, 2022) Yang, Bailin; Wang, Qingxu; Li, Frederick W. B.; Liang, Xiaohui; Wei, Tianxiang; Zhu, Changrui; Yang, Yin; Parakkat, Amal D.; Deng, Bailin; Noh, Seung-Tak
Choosing a suitable color palette can typically improve image aesthetic, where a naive way is choosing harmonious colors from some pre-defined color combinations in color wheels. However, color palettes only consider the usage of color types without specifying their amount in an image. Also, it is still challenging to automatically assign individual palette colors to suitable image regions for maximizing image aesthetic quality. Motivated by these, we propose to construct a contribution-aware color palette from images with high aesthetic quality, enabling color transfer by matching the coloring and regional characteristics of an input image. We hence exploit public image datasets, extracting color composition and embedded color contribution features from aesthetic images to generate our proposed color palettes. We consider both image area ratio and image location as the color contribution features to extract. We have conducted quantitative experiments to demonstrate that our method outperforms existing methods through SSIM (Structural SIMilarity) and PSNR (Peak Signal to Noise Ratio) for objective image quality measurement and no-reference image assessment (NIMA) for image aesthetic scoring.
Album Quickview in Comic-like Layout via Quartet Analysis
(The Eurographics Association, 2014) Zheng, Zhibin; Zhang, Yan; Miao, Zheng; Sun, Zhengxing; John Keyser and Young J. Kim and Peter Wonka
For clear summary and efficient search of images for album, which carries a story of life record, we propose a new approach for quickview of album in comic-like layout via quartet analysis. How to organize the images in album and in what way to display images in collage are two key problems for album quickview. For the first problem, we take the idea of model organization method based on quartet analysis to construct categorization tree to organize the images; while for the second problem, we utilize the topological structure of categorization tree to decompose it into multiple groups of images and extract representative image from each group for subsequent collage. For the collage part, we choose comic-like layout to present collage because comic provides a concise way for storytelling and it has variablitiy in layout styles, which is suitable for album summary. Experiments demonstrate that our method could organize the images effectively and present images in collage with diverse styles as well.
Anisotropic Spectral Manifold Wavelet Descriptor for Deformable Shape Analysis and Matching
(The Eurographics Association, 2018) Li, Qinsong; Liu, Shengjun; Hu, Ling; Liu, Xinru; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
In this paper, we present a novel framework termed Anisotropic Spectral Manifold Wavelet Transform (ASMWT) for shape analysis. ASMWT comprehensively analyzes the signals from multiple directions on local manifold regions of the shape with a series of low-pass and band-pass frequency filters in each direction. Using the ASMWT coefficients of a very simple function, we efficiently construct a localizable and discriminative multiscale point descriptor, named as the Anisotropic Spectral Manifold Wavelet Descriptor (ASMWD). Since the filters used in our descriptor are direction-sensitive and able to robustly reconstruct the signals with a finite number of scales, it makes our descriptor be intrinsic-symmetry unambiguous, compact as well as efficient. The extensive experimental results demonstrate that our method achieves significant performance than several state-of-the-art methods when applied in vertex-wise shape matching.
Art-directing Appearance using an Environment Map Latent Space
(The Eurographics Association, 2021) Petikam, Lohit; Chalmers, Andrew; Anjyo, Ken; Rhee, Taehyun; Lee, Sung-Hee and Zollmann, Stefanie and Okabe, Makoto and Wünsche, Burkhard
In look development, environment maps (EMs) are used to verify 3D appearance in varied lighting (e.g., overcast, sunny, and indoor). Artists can only assign one fixed material, making it laborious to edit appearance uniquely for all EMs. Artists can artdirect material and lighting in film post-production. However, this is impossible in dynamic real-time games and live augmented reality (AR), where environment lighting is unpredictable. We present a new workflow to customize appearance variation across a wide range of EM lighting, for live applications. Appearance edits can be predefined, and then automatically adapted to environment lighting changes. We achieve this by learning a novel 2D latent space of varied EM lighting. The latent space lets artists browse EMs in a semantically meaningful 2D view. For different EMs, artists can paint different material and lighting parameter values directly on the latent space. We robustly encode new EMs into the same space, for automatic look-up of the desired appearance. This solves a new problem of preserving art-direction in live applications, without any artist intervention.
Audio-Driven Speech Animation with Text-Guided Expression
(The Eurographics Association, 2024) Jung, Sunjin; Chun, Sewhan; Noh, Junyong; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
We introduce a novel method for generating expressive speech animations of a 3D face, driven by both audio and text descriptions. Many previous approaches focused on generating facial expressions using pre-defined emotion categories. In contrast, our method is capable of generating facial expressions from text descriptions unseen during training, without limitations to specific emotion classes. Our system employs a two-stage approach. In the first stage, an auto-encoder is trained to disentangle content and expression features from facial animations. In the second stage, two transformer-based networks predict the content and expression features from audio and text inputs, respectively. These features are then passed to the decoder of the pre-trained auto-encoder, yielding the final expressive speech animation. By accommodating diverse forms of natural language, such as emotion words or detailed facial expression descriptions, our method offers an intuitive and versatile way to generate expressive speech animations. Extensive quantitative and qualitative evaluations, including a user study, demonstrate that our method can produce natural expressive speech animations that correspond to the input audio and text descriptions.
Automatic 3D Posing from 2D Hand-Drawn Sketches
(The Eurographics Association, 2014) Gouvatsos, Alexandros; Xiao, Zhidong; Marsden, Neil; Zhang, Jian J.; John Keyser and Young J. Kim and Peter Wonka
Inferring the 3D pose of a character from a drawing is a non-trivial and under-constrained problem. Solving it may help automate various parts of an animation production pipeline such as pre-visualisation. In this paper, a novel way of inferring the 3D pose from a monocular 2D sketch is proposed. The proposed method does not make any external assumptions about the model, allowing it to be used on different types of characters. The 3D pose inference is formulated as an optimisation problem and a parallel variation of the Particle Swarm Optimisation algorithm called PARAC-LOAPSO is utilised for searching the minimum. Testing in isolation as well as part of a larger scene, the presented method is evaluated by posing a lamp and a horse character. The results show that this method is robust and is able to be extended to various types of models.
Automatic Aesthetics-based Lighting Design with Global Illumination
(The Eurographics Association, 2014) Léon, Vincent; Gruson, Adrien; Cozot, Rémi; Bouatouch, Kadi; John Keyser and Young J. Kim and Peter Wonka
In computer graphics, lighting plays an important role in the appearance of a scene. A change in the configuration of light sources can lead to different aesthetics in the final rendered image. Lighting design becomes increasingly complex when using sophisticated global illumination techniques. In this paper, we present a new approach to automatically design the lighting configuration according to the aesthetic goal specified by the user as a set of target parameters. Target parameters are used to set up an objective function which is minimized using an optimization method. The results show that our method can be used to automatically design a lighting configuration that will give to the final image a classic photographic look.
Automatic Garment Modeling From Front And Back Images
(The Eurographics Association, 2014) Huang, Lifeng; Gao, Chengying; John Keyser and Young J. Kim and Peter Wonka
We present a system which can automatically generate a realistic garment model from two images of an existing garment. Without the requirement of tailoring expertise and tedious operation, our method takes the front and back images of a real garment as input, and the system will make reasonable geometric modeling as well as physical simulation of the garment. Combining with mannequin's skeleton information, we propose a panel positioning method to place garment panels in appropriate positions. A key feature of our system is to automatically interpret sewn information, which effectively simplifies user interaction. In addition, panel deformation method based on mannequin's pose allows easy data capture. It extends the flexibility and utility of our method. The experiments demonstrate the effectiveness on generating models of various garment styles.
Automatic Vector Caricature via Face Parametrization
(The Eurographics Association, 2023) Madono, Koki; Hold-Geoffroy, Yannick; Li, Yijun; Ito, Daichi; Echevarria, Jose; Smith, Cameron; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Automatic caricature generation is a challenging task that aims to emphasize the subject's facial characteristics while preserving its identity. Due to the complexity of the task, caricatures could exclusively be performed by a trained artist. Recent developments in deep learning have achieved promising results in capturing artistic styles. Despite the success, current methods still struggle to accurately capture the whimsical aspect of caricatures while preserving identity. In this work, we propose Parametric Caricature, the first parametric-based caricature generation that yields vectorized and animatable caricatures. We devise several hundred parameters to encode facial traits, which our method directly predicts instead of estimating the raster caricature like previous methods. To guide the attention of the method, we segment the different parts of the face and retrieve the most similar parts from an artist-made database of caricatures. Our method proposes visually appealing caricatures more adapted to use as avatars than existing methods, as demonstrated by our user study.
Avatar Emotion Recognition using Non-verbal Communication
(The Eurographics Association, 2023) Bazargani, Jalal Safari; Sadeghi-Niaraki, Abolghasem; Choi, Soo-Mi; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Among the sources of information about emotions, body movements, recognized as ''kinesics'' in non-verbal communication, have received limited attention. This research gap suggests the need to investigate suitable body movement-based approaches for making communication in virtual environments more realistic. Therefore, this study proposes an automated emotion recognition approach suitable for use in virtual environments. This study consists of two pipelines for emotion recognition. For the first pipeline, i.e., upper-body keypoint-based recognition, the HEROES video dataset was employed to train a bidirectional long short-term memory model using upper-body keypoints capable of predicting four discrete emotions: boredom, disgust, happiness, and interest, achieving an accuracy of 84%. For the second pipeline, i.e., wrist-movement-based recognition, a random forest model was trained based on 17 features computed from acceleration data of wrist movements along each axis. The model achieved an accuracy of 63% in distinguishing three discrete emotions: sadness, neutrality, and happiness. The findings suggest that the proposed approach is a noticeable step toward automated emotion recognition, without using any additional sensors other than the head mounted display (HMD).
Backwards Memory Allocation and Improved OIT
(The Eurographics Association, 2013) Knowles, Pyarelal; Leach, Geoff; Zambetta, Fabio; Bruno Levy and Xin Tong and KangKang Yin
Order independent transparency (OIT) is a graphics technique which sorts surfaces per-pixel for correct alpha blending. The sorting stage requires relatively large amounts of temporary memory in shaders that is usually conservatively allocated at a maximum, which impacts occupancy and performance. To address this issue we introduce backwards memory allocation (BMA), a strategy which creates a set of shaders with varying static allocation size in lieu of dynamic allocation. Batches of threads are then executed directly with the appropriate shader. This also allows optimizations for each generated shader such as choosing the sorting algorithm based on allocation size with no additional overhead. BMA gives both a more flexible OIT (BMA-OIT) for dynamic scenes of varying depth complexity and up to a 3x speedup.

Browse

Browsing PG: Pacific Graphics Short Papers by Title

Results Per Page

Sort Options