41-Issue 8

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/2633208

Browse

Now showing 1 - 20 of 31

Learning Physics with a Hierarchical Graph Network
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Chentanez, Nuttapong; Jeschke, Stefan; Müller, Matthias; Macklin, Miles; Dominik L. Michels; Soeren Pirk
We propose a hierarchical graph for learning physics and a novel way to handle obstacles. The finest level of the graph consist of the particles itself. Coarser levels consist of the cells of sparse grids with successively doubling cell sizes covering the volume occupied by the particles. The hierarchical structure allows for the information to propagate at great distance in a single message passing iteration. The novel obstacle handling allows the simulation to be obstacle aware without the need for ghost particles. We train the network to predict effective acceleration produced by multiple sub-steps of 3D multi-material material point method (MPM) simulation consisting of water, sand and snow with complex obstacles. Our network produces lower error, trains up to 7.0X faster and inferences up to 11.3X faster than [SGGP*20]. It is also, on average, about 3.7X faster compared to Taichi Elements simulation running on the same hardware in our tests.
Wassersplines for Neural Vector Field-Controlled Animation
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Zhang, Paul; Smirnov, Dmitriy; Solomon, Justin; Dominik L. Michels; Soeren Pirk
Much of computer-generated animation is created by manipulating meshes with rigs. While this approach works well for animating articulated objects like animals, it has limited flexibility for animating less structured free-form objects. We introduce Wassersplines, a novel trajectory inference method for animating unstructured densities based on recent advances in continuous normalizing flows and optimal transport. The key idea is to train a neurally-parameterized velocity field that represents the motion between keyframes. Trajectories are then computed by advecting keyframes through the velocity field. We solve an additional Wasserstein barycenter interpolation problem to guarantee strict adherence to keyframes. Our tool can stylize trajectories through a variety of PDE-based regularizers to create different visual effects. We demonstrate our tool on various keyframe interpolation problems to produce temporally-coherent animations without meshing or rigging.
Voice2Face: Audio-driven Facial and Tongue Rig Animations with cVAEs
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Villanueva Aylagas, Monica; Anadon Leon, Hector; Teye, Mattias; Tollmar, Konrad; Dominik L. Michels; Soeren Pirk
We present Voice2Face: a Deep Learning model that generates face and tongue animations directly from recorded speech. Our approach consists of two steps: a conditional Variational Autoencoder generates mesh animations from speech, while a separate module maps the animations to rig controller space. Our contributions include an automated method for speech style control, a method to train a model with data from multiple quality levels, and a method for animating the tongue. Unlike previous works, our model generates animations without speaker-dependent characteristics while allowing speech style control. We demonstrate through a user study that Voice2Face significantly outperforms a comparative state-of-the-art model in terms of perceived animation quality, and our quantitative evaluation suggests that Voice2Face yields more accurate lip closure in speech with bilabials through our speech style optimization. Both evaluations also show that our data quality conditioning scheme outperforms both an unconditioned model and a model trained with a smaller high-quality dataset. Finally, the user study shows a preference for animations including tongue. Results from our model can be seen at https://go.ea.com/voice2face.
Cognitive Model of Agent Exploration with Vision and Signage Understanding
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Johnson, Colin; Haworth, Brandon; Dominik L. Michels; Soeren Pirk
Signage systems play an essential role in ensuring safe, stress-free, and efficient navigation for the occupants of indoor spaces. Crowd simulations with sufficiently realistic virtual humans provide a convenient and cost-effective approach to evaluating and optimizing signage systems. In this work, we develop an agent model which makes use of image processing on parametric saliency maps to visually identify signage and distractions in the agent's field of view. Information from identified signs is incorporated into a grid-based representation of wayfinding familiarity, which is used to guide informed exploration of the agent's environment using a modified A* algorithm. In areas with low wayfinding familiarity, the agent follows a random exploration behaviour based on sampling a grid of previously observed locations for heuristic values based on space syntax isovist measures. The resulting agent design is evaluated in a variety of test environments and found to be able to reliably navigate towards a goal location using a combination of signage and random exploration.
Generating Upper-Body Motion for Real-Time Characters Making their Way through Dynamic Environments
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Alvarado, Eduardo; Rohmer, Damien; Cani, Marie-Paule; Dominik L. Michels; Soeren Pirk
Real-time character animation in dynamic environments requires the generation of plausible upper-body movements regardless of the nature of the environment, including non-rigid obstacles such as vegetation. We propose a flexible model for upper-body interactions, based on the anticipation of the character's surroundings, and on antagonistic controllers to adapt the amount of muscular stiffness and response time to better deal with obstacles. Our solution relies on a hybrid method for character animation that couples a keyframe sequence with kinematic constraints and lightweight physics. The dynamic response of the character's upper-limbs leverages antagonistic controllers, allowing us to tune tension/relaxation in the upper-body without diverging from the reference keyframe motion. A new sight model, controlled by procedural rules, enables high-level authoring of the way the character generates interactions by adapting its stiffness and reaction time. As results show, our real-time method offers precise and explicit control over the character's behavior and style, while seamlessly adapting to new situations. Our model is therefore well suited for gaming applications.
Facial Animation with Disentangled Identity and Motion using Transformers
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Chandran, Prashanth; Zoss, Gaspard; Gross, Markus; Gotardo, Paulo; Bradley, Derek; Dominik L. Michels; Soeren Pirk
We propose a 3D+time framework for modeling dynamic sequences of 3D facial shapes, representing realistic non-rigid motion during a performance. Our work extends neural 3D morphable models by learning a motion manifold using a transformer architecture. More specifically, we derive a novel transformer-based autoencoder that can model and synthesize 3D geometry sequences of arbitrary length. This transformer naturally determines frame-to-frame correlations required to represent the motion manifold, via the internal self-attention mechanism. Furthermore, our method disentangles the constant facial identity from the time-varying facial expressions in a performance, using two separate codes to represent neutral identity and the performance itself within separate latent subspaces. Thus, the model represents identity-agnostic performances that can be paired with an arbitrary new identity code and fed through our new identity-modulated performance decoder; the result is a sequence of 3D meshes for the performance with the desired identity and temporal length. We demonstrate how our disentangled motion model has natural applications in performance synthesis, performance retargeting, key-frame interpolation and completion of missing data, performance denoising and retiming, and other potential applications that include full 3D body modeling.
SCA 2022 CGF 41-8: Frontmatter
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Dominik L. Michels; Soeren Pirk; Dominik L. Michels; Soeren Pirk
Voronoi Filters for Simulation Enrichment
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Casafranca, Juan J.; Otaduy, Miguel A.; Dominik L. Michels; Soeren Pirk
The simulation of complex deformation problems often requires enrichment techniques that introduce local high-resolution detail on a generally coarse discretization. The use cases include spatial or temporal refinement of the discretization, the simulation of composite materials with phenomena occurring at different scales, or even codimensional simulation. We present an efficient simulation enrichment method for both local refinement of the discretization and codimensional effects. We dub our method Voronoi filters, as it combines two key computational elements. One is the use of kinematic filters to constrain coarse and fine deformations, and thus provide enrichment functions that are complementary to the coarse deformation. The other one is the use of a centroidal Voronoi discretization for the design of the enrichment functions, which adds high-resolution detail in a compact manner while preserving the rigid modes of coarse deformation. We demonstrate our method on simulation examples of composite materials, hybrid triangle-based and yarn-level simulation of cloth, or enrichment of flesh simulation with high-resolution detail.
Combining Motion Matching and Orientation Prediction to Animate Avatars for Consumer-Grade VR Devices
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Ponton, Jose Luis; Yun, Haoran; Andujar, Carlos; Pelechano, Nuria; Dominik L. Michels; Soeren Pirk
The animation of user avatars plays a crucial role in conveying their pose, gestures, and relative distances to virtual objects or other users. Self-avatar animation in immersive VR helps improve the user experience and provides a Sense of Embodiment. However, consumer-grade VR devices typically include at most three trackers, one at the Head Mounted Display (HMD), and two at the handheld VR controllers. Since the problem of reconstructing the user pose from such sparse data is ill-defined, especially for the lower body, the approach adopted by most VR games consists of assuming the body orientation matches that of the HMD, and applying animation blending and time-warping from a reduced set of animations. Unfortunately, this approach produces noticeable mismatches between user and avatar movements. In this work we present a new approach to animate user avatars that is suitable for current mainstream VR devices. First, we use a neural network to estimate the user's body orientation based on the tracking information from the HMD and the hand controllers. Then we use this orientation together with the velocity and rotation of the HMD to build a feature vector that feeds a Motion Matching algorithm. We built a MoCap database with animations of VR users wearing a HMD and used it to test our approach on both self-avatars and other users' avatars. Our results show that our system can provide a large variety of lower body animations while correctly matching the user orientation, which in turn allows us to represent not only forward movements but also stepping in any direction.
A Fusion Crowd Simulation Method: Integrating Data with Dynamics, Personality with Common
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Mao, Tianlu; Wang, Ji; Meng, Ruoyu; Yan, Qinyuan; Liu, Shaohua; Wang, Zhaoqi; Dominik L. Michels; Soeren Pirk
This paper proposes a novel crowd simulation method which integrates not only modelling ideas but also advantages from both data-driven methods and crowd dynamics methods. To seamlessly integrate these two different modelling ideas, first, a fusion crowd motion model is developed. In this model the motion of crowd are driven dynamically by different forces. Part of the forces are modeled under a universal interaction mechanism, which describe the common parts of crowd dynamics. Others are modeled by examples from real data, which describe the personality parts of the agent motion. Second, a construction method for example dataset is proposed to support the fusion model. In the dataset, crowd trajectories captured in the real world are decomposed and re-described under the structure of the fusion model. Thus, personality parts hidden in the real data could be locked and extracted, making the data understandable and migratable for our fusion model. A comprehensive crowd motion generation workflow using the fusion model and example dataset is also proposed. Quantitative and qualitative experiments and user studies are conducted. Results show that the proposed fusion crowd simulation method can generate crowd motion with the great motion fidelity, which not only match the macro characteristics of real data, but also has lots of micro personality showing the diversity of crowd motion.
High-Order Elasticity Interpolants for Microstructure Simulation
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Chan-Lock, Antoine; Pérez, Jesús; Otaduy, Miguel A.; Dominik L. Michels; Soeren Pirk
We propose a novel formulation of elastic materials based on high-order interpolants, which fits accurately complex elastic behaviors, but remains conservative. The proposed high-order interpolants can be regarded as a high-dimensional extension of radial basis functions, and they allow the interpolation of derivatives of elastic energy, in particular stress and stiffness. Given the proposed parameterization of elasticity models, we devise an algorithm to find optimal model parameters based on training data. We have tested our methodology for the homogenization of 2D microstructures, and we show that it succeeds to match complex behaviors with high accuracy.
Sketching Vocabulary for Crowd Motion
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Mathew, C. D. Tharindu; Benes, Bedrich; Aliaga, Daniel; Dominik L. Michels; Soeren Pirk
This paper proposes and evaluates a sketching language to author crowd motion. It focuses on the path, speed, thickness, and density parameters of crowd motion. A sketch-based vocabulary is proposed for each parameter and evaluated in a user study against complex crowd scenes. A sketch recognition pipeline converts the sketches into a crowd simulation. The user study results show that 1) participants at various skill levels and can draw accurate crowd motion through sketching, 2) certain sketch styles lead to a more accurate representation of crowd parameters, and 3) sketching allows to produce complex crowd motions in a few seconds. The results show that some styles although accurate actually are less preferred over less accurate ones.
Neural3Points: Learning to Generate Physically Realistic Full-body Motion for Virtual Reality Users
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Ye, Yongjing; Liu, Libin; Hu, Lei; Xia, Shihong; Dominik L. Michels; Soeren Pirk
Animating an avatar that reflects a user's action in the VR world enables natural interactions with the virtual environment. It has the potential to allow remote users to communicate and collaborate in a way as if they met in person. However, a typical VR system provides only a very sparse set of up to three positional sensors, including a head-mounted display (HMD) and optionally two hand-held controllers, making the estimation of the user's full-body movement a difficult problem. In this work, we present a data-driven physics-based method for predicting the realistic full-body movement of the user according to the transformations of these VR trackers and simulating an avatar character to mimic such user actions in the virtual world in realtime. We train our system using reinforcement learning with carefully designed pretraining processes to ensure the success of the training and the quality of the simulation. We demonstrate the effectiveness of the method with an extensive set of examples.
Pose Representations for Deep Skeletal Animation
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Andreou, Nefeli; Aristidou, Andreas; Chrysanthou, Yiorgos; Dominik L. Michels; Soeren Pirk
Data-driven skeletal animation relies on the existence of a suitable learning scheme, which can capture the rich context of motion. However, commonly used motion representations often fail to accurately encode the full articulation of motion, or present artifacts. In this work, we address the fundamental problem of finding a robust pose representation for motion, suitable for deep skeletal animation, one that can better constrain poses and faithfully capture nuances correlated with skeletal characteristics. Our representation is based on dual quaternions, the mathematical abstractions with well-defined operations, which simultaneously encode rotational and positional orientation, enabling a rich encoding, centered around the root. We demonstrate that our representation overcomes common motion artifacts, and assess its performance compared to other popular representations. We conduct an ablation study to evaluate the impact of various losses that can be incorporated during learning. Leveraging the fact that our representation implicitly encodes skeletal motion attributes, we train a network on a dataset comprising of skeletons with different proportions, without the need to retarget them first to a universal skeleton, which causes subtle motion elements to be missed. Qualitative results demonstrate the usefulness of the parameterization in skeleton-specific synthesis.
Physically Based Shape Matching
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Müller, Matthias; Macklin, Miles; Chentanez, Nuttapong; Jeschke, Stefan; Dominik L. Michels; Soeren Pirk
The shape matching method is a popular approach to simulate deformable objects in interactive applications due to its stability and simplicity. An important feature is that there is no need for a mesh since the method works on arbitrary local groups within a set of particles. A major drawback of shape matching is the fact that it is geometrically motivated and not derived from physical principles which makes calibration difficult. The fact that the method does not conserve volume can yield visual artifacts, e.g. when a tire is compressed but does not bulge. In this paper we present a new meshless simulation method that is related to shape matching but derived from continuous constitutive models. Volume conservation and stiffness can be specified with physical parameters. Further, if the elements of a tetrahedral mesh are used as groups, our method perfectly reproduces FEM based simulations.
Detailed Eye Region Capture and Animation
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Kerbiriou, Glenn; Marchal, Maud; Avril, Quentin; Dominik L. Michels; Soeren Pirk
Even if the appearance and geometry of the human eye have been extensively studied during the last decade, the geometrical correlation between gaze direction, eyelids aperture and eyelids shape has not been empirically modeled. In this paper, we propose a data-driven approach for capturing and modeling the subtle features of the human eye region, such as the inner eye corner and the skin bulging effect due to globe orientation. Our approach consists of an original experimental setup to capture the eye region geometry variations combined with a 3D reconstruction method. Regarding the eye region capture, we scanned 55 participants doing 36 eyes poses. To animate a participant's eye region, we register the different poses to a vertex wise correspondence before blending them in a trilinear fashion. We show that our 3D animation results are visually pleasant and realistic while bringing novel eye features compared to state of the art models.
Differentiable Simulation for Outcome-Driven Orthognathic Surgery Planning
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Dorda, Daniel; Peter, Daniel; Borer, Dominik; Huber, Niko Benjamin; Sailer, Irena; Gross, Markus; Solenthaler, Barbara; Thomaszewski, Bernhard; Dominik L. Michels; Soeren Pirk
Algorithms at the intersection of computer graphics and medicine have recently gained renewed attention. A particular interest are methods for virtual surgery planning (VSP), where treatment parameters must be carefully chosen to achieve a desired treatment outcome. FEM simulators can verify the treatment parameters by comparing a predicted outcome to the desired one. However, estimating the optimal parameters amounts to solving a challenging inverse problem. In current clinical practice it is solved manually by surgeons, who rely on their experience and intuition to iteratively refine the parameters, verifying them with simulated predictions. We prototype a differentiable FEM simulator and explore how it can enhance and simplify treatment planning, which is ultimately necessary to integrate simulation-based VSP tools into a clinical workflow. Specifically, we define a parametric treatment model based on surgeon input, and with analytically derived simulation gradients we optimise it against an objective defined on the visible facial 3D surface. By using sensitivity analysis, we can easily explore the solution-space with first-order approximations, which allow the surgeon to interactively visualise the effect of parameter variations on a given treatment plan. The objective function allows landmarks to be freely chosen, accommodating the multiple methodologies in clinical planning. We show that even with a very sparse set of guiding landmarks, our simulator robustly converges to a feasible post-treatment shape.
Fast Numerical Coarsening with Local Factorizations
(The Eurographics Association and John Wiley & Sons Ltd., 2022) He, Zhongyun; Pérez, Jesús; Otaduy, Miguel A.; Dominik L. Michels; Soeren Pirk
Numerical coarsening methods offer an attractive methodology for fast simulation of objects with high-resolution heterogeneity. However, they rely heavily on preprocessing, and are not suitable when objects undergo dynamic material or topology updates. We present methods that largely accelerate the two main processes of numerical coarsening, namely training data generation and the optimization of coarsening shape functions, and as a result we manage to leverage runtime numerical coarsening under local material updates. To accelerate the generation of training data, we propose a domain-decomposition solver based on substructuring that leverages local factorizations. To accelerate the computation of coarsening shape functions, we propose a decoupled optimization of smoothness and data fitting. We evaluate quantitatively the accuracy and performance of our proposed methods, and we show that they achieve accuracy comparable to the baseline, albeit with speed-ups of orders of magnitude. We also demonstrate our methods on example simulations with local material and topology updates.
Context-based Style Transfer of Tokenized Gestures
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Kuriyama, Shigeru; Mukai, Tomohiko; Taketomi, Takafumi; Mukasa, Tomoyuki; Dominik L. Michels; Soeren Pirk
Gestural animations in the amusement or entertainment field often require rich expressions; however, it is still challenging to synthesize characteristic gestures automatically. Although style transfer based on a neural network model is a potential solution, existing methods mainly focus on cyclic motions such as gaits and require re-training in adding new motion styles. Moreover, their per-pose transformation cannot consider the time-dependent features, and therefore motion styles of different periods and timings are difficult to be transferred. This limitation is fatal for the gestural motions requiring complicated time alignment due to the variety of exaggerated or intentionally performed behaviors. This study introduces a context-based style transfer of gestural motions with neural networks to ensure stable conversion even for exaggerated, dynamically complicated gestures. We present a model based on a vision transformer for transferring gestures' content and style features by time-segmenting them to compose tokens in a latent space. We extend this model to yield the probability of swapping gestures' tokens for style-transferring. A transformer model is suited to semantically consistent matching among gesture tokens, owing to the correlation with spoken words. The compact architecture of our network model requires only a small number of parameters and computational costs, which is suitable for real-time applications with an ordinary device. We introduce loss functions provided by the restoration error of identically and cyclically transferred gesture tokens and the similarity losses of content and style evaluated by splicing features inside the transformer. This design of losses allows unsupervised and zero-shot learning, by which the scalability for motion data is obtained. We comparatively evaluated our style transfer method, mainly focusing on expressive gestures using our dataset captured for various scenarios and styles by introducing new error metrics tailored for gestures. Our experiment showed the superiority of our method in numerical accuracy and stability of style transfer against the existing methods.
Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi-Hot Class Embedding
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Goel, Aman; Men, Qianhui; Ho, Edmond S. L.; Dominik L. Michels; Soeren Pirk
Synthesizing multi-character interactions is a challenging task due to the complex and varied interactions between the characters. In particular, precise spatiotemporal alignment between characters is required in generating close interactions such as dancing and fighting. Existing work in generating multi-character interactions focuses on generating a single type of reactive motion for a given sequence which results in a lack of variety of the resultant motions. In this paper, we propose a novel way to create realistic human reactive motions which are not presented in the given dataset by mixing and matching different types of close interactions. We propose a Conditional Hierarchical Generative Adversarial Network with Multi-Hot Class Embedding to generate the Mix and Match reactive motions of the follower from a given motion sequence of the leader. Experiments are conducted on both noisy (depth-based) and high-quality (MoCap-based) interaction datasets. The quantitative and qualitative results show that our approach outperforms the state-of-the-art methods on the given datasets. We also provide an augmented dataset with realistic reactive motions to stimulate future research in this area.

Browse

Browsing 41-Issue 8 by Issue Date

Results Per Page

Sort Options