2015
Permanent URI for this collection
Scalable Exploration of Highly Detailed and Annotated 3D Models
[meta data] [files:
Balsa Rodríguez, Marcos
Path Sampling Techniques for Efficient Light Transport Simulation
[meta data] [files:
Georgiev, Iliyan
Capacitive Sensing and Communication for Ubiquitous Interaction and Environmental Perception
[meta data] [files:
Grosse-Puppendahl, Tobias
Sketching free-form poses and movements for expressive character animation
[meta data] [files:
Guay, Martin
Using natural user interfaces to support learning environments
[meta data] [files:
Martín-SanJosé, Juan-Fernando
Supporting Management of Sensor Networks through Interactive Visual Analysis
[meta data] [files:
Steiger, Martin
Computer Drawing Tools for Assisting Learners, Hobbyists, and Professionals
[meta data] [files:
Iarussi, Emmanuel
Sampling and Variance Analysis for Monte Carlo Integration in Spherical Domain
[meta data] [files:
Singh, Gurprit
3D Reconstruction and Rendering from High Resolution Light Fields
[meta data] [files:
Kim, Changil
A Framework for Local Terrain Deformation Based on Diffusion Theory
[meta data] [files:
Gilardi, Marco
Interactive Integrated Exploration and Management of Visualization Parameters
[meta data] [files:
Mindek, Peter
Scheduling activities under spatial and temporal constraints to populate virtual urban environments
[meta data] [files:
Jorgensen, Carl-Johan
Reinforcement learning in a Multi-agent Framework for Pedestrian Simulation
[meta data] [files:
Martinez-Gil, Francisco
Marker-less Motion Capture in General Scenes with Sparse Multi-camera Setups
[meta data] [files:
Elhayek, Ahmed
Printing Beyond Color: Spectral and Specular Reproduction
[meta data] [files:
Samadzadegan, Sepideh
Physically based rendering of synthetic objects in real environments
[meta data] [files:
Joel, Kronander
Browse
Recent Submissions
Item Illustrative Visualization of Medical Data Sets(2015-01-29) Lawonn, KaiThe aim of an illustrative visualization method is to provide a simplified representation of a complex scene or object. Concave and convex regions are emphasized and the surface complexity is reduced by omitting unnecessary information. This abstraction is often preferred over fully illuminated scenes in a multitude of applications. This thesis presents an overview of the state-of-the-art for feature lines. For this, the mathematical background of the differential geometry will be presented. Furthermore, this background will be adapted to triangulated surface meshes. This thesis will also present a smoothing algorithm, which smooths initial curves on triangulated surfaces. Additionally, an evaluation is shown to assess the quality of feature lines on anatomical data sets. Based on the evaluation, two conclusions in the field for medical applications were derived. From this point, this thesis presents two solutions in the field of illustrative visualization for medical data sets. A novel line drawing technique will be presented to illustrate surfaces. According to different requirements, this technique will be extended for molecular surfaces. In the field of vessel visualization with embedded blood flow, an adaptive visualization method will be introduced. This technique will also be extended to animated blood flow. Finally, this thesis shows different illustrative visualization concepts, which can be applied in various fields for depicting surface information.Item Scalable Exploration of Highly Detailed and Annotated 3D Models(2015) Balsa Rodríguez, MarcosWith the widespread availability of mobile graphics terminals and WebGL-enabled browsers, 3D graphics over the Internet is thriving. Thanks to recent advances in 3D acquisition and modeling systems, high-quality 3D models are becoming increasingly common, and are now potentially available for ubiquitous exploration. In current 3D repositories, such as Blend Swap, 3D Café or Archive3D, 3D models available for download are mostly presented through a few user-selected static images. Online exploration is limited to simple orbiting and/or low-fidelity explorations of simplified models, since photo- realistic rendering quality of complex synthetic environments is still hardly achievable within the real-time constraints of interactive applications, especially on on low-powered mobile devices or script-based Internet browsers. Moreover, navigating inside 3D environments, especially on the now pervasive touch devices, is a non-trivial task, and usability is consistently improved by employing assisted navigation controls. In addition, 3D annotations are often used in order to integrate and enhance the visual information by providing spatially coherent contextual information, typically at the expense of introducing visual cluttering. In this thesis, we focus on e cient representations for interactive exploration and understand- ing of highly detailed 3D meshes on common 3D platforms. For this purpose, we present several approaches exploiting constraints on the data representation for improving the streaming and rendering performance, and camera movement constraints in order to provide scalable navigation methods for interactive exploration of complex 3D environments. Furthermore, we study visualization and interaction techniques to improve the exploration and understanding of complex 3D models by exploiting guided motion control techniques to aid the user in discovering contextual information while avoiding cluttering the visualization. We demonstrate the e ectiveness and scalability of our approaches both in large screen museum installations and in mobile devices, by performing interactive exploration of models ranging from 9Mtriangles to 940Mtriangles.Item Path Sampling Techniques for Efficient Light Transport Simulation(2015-06) Georgiev, IliyanReproducing the interactions between light and matter in a physically accurate way can significantly improve the realistic appearance of synthetic images, however such effects can be very computationally expensive to simulate. Pressed by strict requirements on image quality and visual realism, industrial applications have recently moved away from using legacy rasterization-based rendering solutions to fully embrace physically-based Monte Carlo methods. This dramatic shift has rekindled the interest in developing new and robust light transport simulation algorithms that can efficiently handle a wide range of scenes with complex materials and lighting – a problem that we address in this thesis. State-of-the-art Monte Carlo methods solve the global illumination problem by sampling random light transport paths in the scene via ray tracing. We analyze the efficiency of these methods, devise new path sampling techniques for rendering surface and volumetric light scattering, and develop novel means of leveraging illumination coherence via path reuse. This results in several practical rendering algorithms that produce images with less noise and remain more resilient to variations in the scene configuration than existing methods. The improved efficiency of these algorithms comes from the use of new and diverse sampling techniques, each specialized for handling a different set of lighting effects. Their robustness is due to the adaptive combination of these techniques in a way that preserves their individual strengths.Item Development and Improvement of Optical Tracking Methods towards Registering the Deformations of 3D Non- Rigid Bodies in Real Time for Augmented Reality Applications(Servicio de Publicaciones. Univesidad de Navarra, 2015-03-27) Leizea, IbaiAugmented Reality (AR) is a technology that aims to embed virtual objects in the real world, showing to the user the set of objects (virtual and real) as a single world. For that purpose, it is necessary to offer a perfect alignment between virtual and real objects, which increases the effectiveness of AR. The solution to this problem is known as tracking. The object tracking consists in determining at any time the position and orientation of the camera relative to the scene. Optical sensors are most commonly used to overcome the tracking problem due to their low cost implementation. However, it is often difficult to provide robustness, accuracy and low computational cost at the same time. This thesis tackles the improvement and development of the main optical tracking techniques, primarily focused on detecting the deformations of the bodies. First, it has been achieved the tracking of rigid and non-rigid planar surfaces through a monocular camera, and then, the object deformation estimation with a more complex device as a RGB-D camera has been developed. Surface tracking systems such as those based on markers have the problem of not being able to handle occlusions. Thus, this thesis raises a new marker design that offers robustness against occlusions. Furthermore, in order to handle the deformations of surfaces, a solution that recovers the camera pose and the non-rigid surface simultaneously is proposed. Continuing with the deformation handling, it has also developed a robust tracking system for reconstructing the 3D shape of deformable objects using two different physical formulations. One offers a correct physical behaviour with a low computational cost, whereas the other achieves higher levels of accuracy at the expense of higher processing time. In addition, all the presented solutions have the common factor that all are executed in real time, which is a key property for a fluently visual feedback of an AR application.Item Capacitive Sensing and Communication for Ubiquitous Interaction and Environmental Perception(2015-06-08) Grosse-Puppendahl, TobiasDuring the last decade, the functionalities of electronic devices within a living environment constantly increased. Besides the personal computer, now tablet PCs, smart household appliances, and smartwatches enriched the technology landscape. The trend towards an ever-growing number of computing systems has resulted in many highly heterogeneous human-machine interfaces. Users are forced to adapt to technology instead of having the technology adapt to them. Gathering context information about the user is a key factor for improving the interaction experience. Emerging wearable devices show the benefits of sophisticated sensors which make interaction more efficient, natural, and enjoyable. However, many technologies still lack of these desirable properties, motivating me to work towards new ways of sensing a user's actions and thus enriching the context. In my dissertation I follow a human-centric approach which ranges from sensing hand movements to recognizing whole-body interactions with objects. This goal can be approached with a vast variety of novel and existing sensing approaches. I focused on perceiving the environment with quasi-electrostatic fields by making use of capacitive coupling between devices and objects. Following this approach, it is possible to implement interfaces that are able to recognize gestures, body movements and manipulations of the environment at typical distances up to 50cm. These sensors usually have a limited resolution and can be sensitive to other conductive objects or electrical devices that affect electric fields. The technique allows for designing very energy-efficient and high-speed sensors that can be deployed unobtrusively underneath any kind of non-conductive surface. Compared to other sensing techniques, exploiting capacitive coupling also has a low impact on a user's perceived privacy. In this work, I also aim at enhancing the interaction experience with new perceptional capabilities based on capacitive coupling. I follow a bottom-up methodology and begin by presenting two low-level approaches for environmental perception. In order to perceive a user in detail, I present a rapid prototyping toolkit for capacitive proximity sensing. The prototyping toolkit shows significant advancements in terms of temporal and spatial resolution. Due to some limitations, namely the inability to determine the identity and fine-grained manipulations of objects, I contribute a generic method for communications based on capacitive coupling. The method allows for designing highly interactive systems that can exchange information through air and the human body. I furthermore show how human body parts can be recognized from capacitive proximity sensors. The method is able to extract multiple object parameters and track body parts in real-time. I conclude my thesis with contributions in the domain of context-aware devices and explicit gesture-recognition systems.Item Sketching free-form poses and movements for expressive character animation(2015-07-02) Guay, MartinFree-form animation allows for exaggerated and artistic styles of motions such as stretching character limbs and animating imaginary creatures such as dragons. Creating these animations requires tools flexible enough to shape characters into arbitrary poses, and control motion at any instant in time. The current approach to free-form animation is keyframing: a manual task in which animators deform characters at individual instants in time by clicking-and-dragging individual body parts one at a time. While this approach is flexible, it is challenging to create quality animations that follow high-level artistic principles—as keyframing tools only provide localized control both spatially and temporally. When drawing poses and motions, artists rely on different sketch-based abstractions that help fulfill high-level aesthetic and artistic principles. For instance, animators will draw lines of action to create more readable and expressive poses. To coordinate movements, animators will sketch motion abstractions such as semi-circles and loops to coordinate bouncing and rolling motions. Unfortunately, these drawing tools are not part of the free-form animation tool set today. The fact that we cannot use the same artistic tools for drawing when animating 3D characters has an important consequence: 3D animation tools are not involved in the creative process. Instead, animators create by first drawing on paper, and only later are 3D animation tools used to fulfill the pose or animation. The reason we do not have these artistic tools (the line of action, and motion abstractions) in the current animation tool set is because we lack a formal understanding relating the character’s shape—possibly over time—to the drawn abstraction’s shape. Hence the main contribution of this thesis is a formal understanding of pose and motion abstractions (line of action and motion abstractions) together with a set of algorithms that allow using these tools in a free-form setting. As a result, the techniques described in this thesis allow exaggerated poses and movements that may include squash and stretch, and can be used with various character morphologies. These pose and animation drafting tools can be extended. For instance, an animator can sketch and compose different layers of motion on top of one another, add twist around strokes, or turning the strokes into elastic ribbons.Item Using natural user interfaces to support learning environments(Riunet, 2015-09-24) Martín-SanJosé, Juan-FernandoEducation is a field of research in which Natural User Interfaces (NUI) have not been extensively exploited. NUI can help in the learning process, specially when used by children. Nowadays, children are growing up playing with computer games, using mobile devices, and other technological devices. New learning methods that use new technologies can help in the learning process. The use of autostereoscopic vision is rising. At present, we can find many devices capables of rendering images that can give us a 3D perception without the use of glasses or any other wearable device. With this technology, new applications with more immersive environments can be developed and brought it to user's disposal. Considering the importance of games and new technologies for learning, in this thesis, two different systems that use NUI for learning about a period of history were designed and developed. One of these systems uses autostereoscopic visualization, which lets the children see themselves as a background in the game, and that renders the elements with 3D sensation without the need for wearing special glasses or other devices. The other system uses frontal projection over a large-size tabletop display for visualization. The two systems have been developed from scratch. The Microsoft Kinect device is used in both systems for interaction. A total of five studies were carried out to determine the efficacy of games with NUI interaction with regard to acquiring knowledge, ease of use, satisfaction, fun and engagement, and their influence on children. In the first study, a comparison of the autostereoscopic system with the frontal projected system was carried out. This study analyzed different aspects such as engagement, increase of knowledge, or preferences. A total of 162 children from 8 to 11 years old participated in the study. From the results, we observed that the different characteristics of the systems did not influence the children's acquired knowledge, engagement, or satisfaction; we also observed that the systems are specially suitable for boys and older children (9-11 years old). The children had the depth perception with the autostereoscopic system. The children considered the two systems easy to use. However, they found the frontal projection to be easier to use. A second comparative study was performed to determine the mode in which the children learn more about the topic of the game. The two modes compared were the collaborative mode, where the children played with the game in pairs; and the individual mode, where the children played with the game solo. A total of 46 children from 7 to 10 years old participated in this study. From the results, we observed that there were statistically significant differences between playing with the game in the two modes. The children who played with the game in pairs in the collaborative mode got better knowledge scores than children who played with the game individually. A third study that compares traditional learning with a collaborative learning method (in pairs and in large groups) using the game was carried out. A total of 100 children from 8 to 10 years old participated in this study. The results are in line with the second study. The children obtained higher score when collaborated in large groups or in pairs than attending to a traditional class. There were no statistically significant differences between playing in large groups and playing in pairs. For personalized learning, a Free Learning Itinerary has been included, where the children can decide how to direct the flow of their own learning process. For comparison, a Linear Learning Itinerary has also been included, where the children follow a determined learning flow. A fourth study to compare the two different learning itineraries was carried out. A total of 29 children from 8 to 9 years old participated in this fourth study. The results showed that there were no statistically significant differences between the two learning itineraries. Regarding the online formative assessment and multiple-choice questions, there is usually a question and several possible answers in questionnaires of this kind in which the student must select only one answer. It is very common for the answers to be just text. However, images could also be used. We have carried out a study to determine if an added image that represents/defines an object helps the children to choose the correct answer. A total of 94 children from 7 to 8 years old participated in the study. The children who filled out the questionnaires with imaged obtained higher score than the children who filled out the text-only questionnaire. No statistically significant differences were found between the two questionnaire types with images. The results from the studies suggest that games of this kind could be appropriate educational games, and that autostereoscopy is a technology to exploit in their development. The following general conclusions have been extracted from the studies carried out: - The new technologies are appropriated for developing educational games and autostereoscopy is a technology to exploit in their development. - With applications like the ones presented in this thesis, children can learn using new technologies and, at the same time, they can have a good time playing the game, which promotes their learning. - To play in a collaborative way facilitates the effectiveness of games with educational purposes. Children can learn a wide variety of educational topics by using new technologies and having fun with other children at the same time they are playing with them.Item Data-driven Modelling of Shape Structure(2015-09) Averkiou, MelinosIn recent years, the study of shape structure has shown great promise, by taking steps towards exposing shape semantics and functionality to algorithms spanning a wide range of areas in computer graphics and vision. By shape structure, we refer to the set of parts that make a shape, the relations between these parts, and the ways in which they correspond and vary between shapes of the same family. These developments have been largely driven by the abundance of 3D data, with collections of 3D models becoming increasingly prominent and websites such as Trimble 3D Warehouse offering millions of free 3D models to the public. The ability to use large amounts of data inside these shape collections for discovering shape structure has made novel approaches to acquisition, modelling, fabrication, and recognition of 3D objects possible. Discovering and modelling the structure of shapes using such data is therefore of great importance. In this thesis we address the problem of discovering and modelling shape structure from large, diverse and unorganized shape collections. Our hypothesis is that by using the large amounts of data inside such shape collections we can discover and model shape structure, and thus use such information to enable structure-aware tools for 3D modelling, including shape exploration, synthesis and editing. We make three key contributions. First, we propose an efficient algorithm for co-aligning large and diverse collections of shapes, to tackle the first challenge in detecting shape structure, which is to place shapes in a common coordinate frame. Then, we introduce a method to parameterize shapes in terms of locations and sizes of their parts, and we demonstrate its application to concurrently exploring a shape collection and synthesizing new shapes. Finally, we define a meta-representation for a shape family, which models the relations of shape parts to capture the main geometric characteristics of the family, and we demonstrate how it can be used to explore shape collections and intelligently edit shapes.Item Supporting Management of Sensor Networks through Interactive Visual Analysis(2015-07-14) Steiger, MartinWith the increasing capabilities of measurement devices and computing machines, the amount of recorded data grows rapidly. It is so high that manual processing is no longer feasible. The Visual Analytics approach is powerful because it combines the strengths of human recognition and vision system with today’s computing power. Different, but strongly linked visualizations and views provide unique perspectives on the same data elements. The views are linked using position on the screen as well as color, which also plays a secondary role in indicating the degree of similarity. This enables the human recognition system to identify trends and anomalies in a network of measurement readings. As a result, the data analyst has the ability to approach more complex questions such as: are there anomalies in the measurement records? What does the network usually look like? In this work we propose a collection of Visual Analytics approaches to support the user in exploratory search and related tasks in graph data sets. One aspect is graph navigation, where we use the information of existing labels to support the user in analyzing with this data set. Another consideration is the preservation of the user’s mental map, which is supported by smooth transitions between individual keyframes. The later chapters focus on sensor networks, a type of graph data that additionally contains time series data on a per-node basis; this adds an extra dimension of complexity to the problem space. This thesis contributes several techniques to the scientific community in different domains and we summarize them as follows. We begin with an approach for network exploration. This forms the basis for subsequent contributions, as it to supports user in the orientation and the navigation in any kind of network structure. This is achieved by providing a showing only a small subset of the data (in other words: a local graph view). The user expresses interest in a certain area by selecting one of more focus nodes that define the visible subgraph. Visual cues in the form of pointing arrows indicate other areas of the graph that could be relevant for the user. Based on this network exploration paradigm, we present a combination of different techniques that stabilize the layout of such local graph views by reducing acting forces. As a result, the movement of nodes in the node-link diagram is reduced, which reduces the mental effort to track changes on the screen. However, up to this point the approach suffers from one of the most prominent shortcomings of force-directed graph layouts. Little changes in the initial setup, force parameters, or graph topology changes have a strong impact on the visual representation of the drawing. When the user explores the network, the set of visible nodes continuously changes and therefore the layout will look different when an area of the graph is visited a second time. This makes it difficult to identify differences or recognize different drawing as equal in terms of topology. We contribute an approach for the deterministic generation of layouts based on pre-computed layout patches that are stitched at runtime. This ensures that even force-directed layouts are deterministic, allowing the analyst to recognize previously explored areas of the graph. In the next step, we apply these rather general purpose concepts from theory in practical applications. One of the most important network category is that of sensor networks, a type of graph data structure where every node is annotated with a time series. Such networks exist in the form of electric grids and other supply networks. In the wake of distributed and localized energy generation, the analysis of these networks becomes more and more important. We present and discuss a multi-view and multi-perspective environment for network analysis of sensor networks that integrates different data sources. It is then extended into a visualization environment that enables the analyst to track the automated analysis of the processing pipeline of an expert system. As a result, the user can verify the correctness of the system and intervene where necessary. One key issue with expert systems, which typically operate on manually written rules, is that they can deal with explicit statements. They cannot grasp terms such as “uncommon” or “anomalous”. Unfortunately, this is often what the domain experts are looking for. We therefore modify and extend the system into an integrated analysis system for the detection of similar patterns in space and in different granularities of time. Its purpose is to obtain an overview of a large system and to identify hot spots and other anomalies. The idea here is to use similar colors to indicate similar patterns in the network. For that, it is vital to be able to rely on the mapping of time series patterns to color. The Colormap-Explorer supports the analysis and comparison of different implementations of 2D color maps to find the best fit for the task. As soon as the domain expert has identified problems in the networks, he or she might want to take countermeasures to improve the network stability. We present an approach that integrates simulation in the process to perform “What-If” analysis based on an underlying simulation framework. Subsequent runs can be compared to quickly identify differences and discover the effect of changes in the network. The approaches that are presented can be utilized in a large variety of applications and application domains. This enables the domain expert to navigate and explore networks, find key elements such as bridges, and detect spurious trends early.Item Computer Drawing Tools for Assisting Learners, Hobbyists, and Professionals(2015-09-29) Iarussi, EmmanuelDrawing is the earliest form of visual depiction. The goal of this thesis is to facilitate and accelerate drawing for amateurs as well as for expert designers and illustrators, employing graphics, image processing and interaction techniques. As this is a broad spectrum to tackle, we identify three specific problems related to drawing, and propose computer tools to help users to overcome the main challenges on each domain. In Chapter 2 we present an interactive drawing tool to help people practice traditional drawing-by-observation techniques. We build this tool supported by a number of techniques to help people gain consciousness of the shapes in a scene. We automatically extract visual guides from a model photograph. The tool's interface displays these information and offer corrective feedback to guide its manual reproduction. In Chapter 3 we propose a method to help designers to enrich their drawings with color and shading. Professionals frequently draw curvature lines to convey bending of smooth surfaces in concept sketches. We exploit this information and extrapolate curvature lines in a rough concept sketch. This extrapolation allows us to recover the intended 3D curvature field and surface normal at each pixel, to create shading and texturing on top of the sketch. In Chapter 4 we propose a tool that helps users to fabricate wire wrapped jewelry. Analogous to physical 3D drawing, this form of handmade jewelry can be seen as taking sketches into the physical world. The presented method assist the user in the main challenges of creating 2D wire-wrapped jewelry from a 2D drawing: decomposing the input into a set of wires, and bending the wires to give them shape.Item Interactive Virtual Cutting of Elastically Deformable Bodies(Technische Universität München, 2015) Wu, JunThis thesis is concerned with computational problems relevant to simulate elastically deformable bodies with changing topology. In particular, I present interactive techniques for the physically-based simulation of cuts in elastic bodies, for the collision detection among these dynamically separating objects, and for the modeling of residual stress in intact soft tissues to simulate the flap shrinkage after cutting. One motivation of this research is to enhance the functionality and performance of surgery simulators, which are becoming accepted in preoperative surgical planning and in surgical skills training, and the potential of which is yet to be fully exploited.Item The Delta Radiance Field(2015-09-11) Franke, Tobias AlexanderThe wide availability of mobile devices capable of computing high fidelity graphics in real-time has sparked a renewed interest in the development and research of Augmented Reality applications. Within the large spectrum of mixed real and virtual elements one specific area is dedicated to produce realistic augmentations with the aim of presenting virtual copies of real existing objects or soon to be produced products. Surprisingly though, the current state of this area leaves much to be desired: Augmenting objects in current systems are often presented without any reconstructed lighting whatsoever and therefore transfer an impression of being glued over a camera image rather than augmenting reality. In light of the advances in the movie industry, which has handled cases of mixed realities from one extreme end to another, it is a legitimate question to ask why such advances did not fully reflect onto Augmented Reality simulations as well. Generally understood to be real-time applications which reconstruct the spatial relation of real world elements and virtual objects, Augmented Reality has to deal with several uncertainties. Among them, unknown illumination and real scene conditions are the most important. Any kind of reconstruction of real world properties in an ad-hoc manner must likewise be incorporated into an algorithm responsible for shading virtual objects and transferring virtual light to real surfaces in an ad-hoc fashion. The immersiveness of an Augmented Reality simulation is, next to its realism and accuracy, primarily dependent on its responsiveness. Any computation affecting the final image must be computed in real-time. This condition rules out many of the methods used for movie production. The remaining real-time options face three problems: The shading of virtual surfaces under real natural illumination, the relighting of real surfaces according to the change in illumination due to the introduction of a new object into a scene, and the believable global interaction of real and virtual light. This dissertation presents contributions to answer the problems at hand. Current state-of-the-art methods build on Differential Rendering techniques to fuse global illumination algorithms into AR environments. This simple approach has a computationally costly downside, which limits the options for believable light transfer even further. This dissertation explores new shading and relighting algorithms built on a mathematical foundation replacing Differential Rendering. The result not only presents a more efficient competitor to the current state-of-the-art in global illumination relighting, but also advances the field with the ability to simulate effects which have not been demonstrated by contemporary publications until now.Item Data-driven Approaches for Interactive Appearance Editing(2015-06-22) Nguyen, Chuong H.This thesis proposes several techniques for interactive editing of digital content and fast rendering of virtual 3D scenes. Editing of digital content - such as images or 3D scenes - is difficult, requires artistic talent and technical expertise. To alleviate these difficulties, we exploit data-driven approaches that use the easily accessible Internet data (e. g., images, videos, materials) to develop new tools for digital content manipulation. Our proposed techniques allow casual users to achieve high-quality editing by interactively exploring the manipulations without the need to understand the underlying physical models of appearance. First, the thesis presents a fast algorithm for realistic image synthesis of virtual 3D scenes. This serves as the core framework for a new method that allows artists to fine tune the appearance of a rendered 3D scene. Here, artists directly paint the final appearance and the system automatically solves for the material parameters that best match the desired look. Along this line, an example-based material assignment approach is proposed, where the 3D models of a virtual scene can be "materialized" simply by giving a guidance source (image/video). Next, the thesis proposes shape and color subspaces of an object that are learned from a collection of exemplar images. These subspaces can be used to constrain image manipulations to valid shapes and colors, or provide suggestions for manipulations. Finally, data-driven color manifolds which contain colors of a specific context are proposed. Such color manifolds can be used to improve color picking performance, color stylization, compression or white balancing.Item Realtime Face Tracking and Animation(2015) Bouaziz, SofienCapturing and processing human geometry, appearance, and motion is at the core of computer graphics, computer vision, and human-computer interaction. The high complexity of human geometry and motion dynamics, and the high sensitivity of the human visual system to variations and subtleties in faces and bodies make the 3D acquisition and reconstruction of humans in motion a challenging task. Digital humans are often created through a combination of 3D scanning, appearance acquisition, and motion capture, leading to stunning results in recent feature films. However, these methods typically require complex acquisition systems and substantial manual post-processing. As a result, creating and animating high-quality digital avatars entails long turn-around times and substantial production costs. Recent technological advances in RGB-D devices, such as Microsoft Kinect, brought new hopes for realtime, portable, and affordable systems allowing to capture facial expressions as well as hand and body motions. RGB-D devices typically capture an image and a depth map. This permits to formulate the motion tracking problem as a 2D/3D non-rigid registration of a deformable model to the input data. We introduce a novel face tracking algorithm that combines geometry and texture registration with pre-recorded animation priors in a single optimization. This led to unprecedented face tracking quality on a low cost consumer level device. The main drawback of this approach in the context of consumer applications is the need for an offline user-specific training. Robust and efficient tracking is achieved by building an accurate 3D expression model of the user’s face who is scanned in a predefined set of facial expressions. We extended this approach removing the need of a user-specific training or calibration, or any other form of manual assistance, by modeling online a 3D user-specific dynamic face model. In complement of a realtime face tracking and modeling algorithm, we developed a novel system for animation retargeting that allows learning a high-quality mapping between motion capture data and arbitrary target characters. We addressed one of the main challenges of existing example-based retargeting methods, the need for a large number of accurate training examples to define the correspondence between source and target expression spaces. We showed that this number can be significantly reduced by leveraging the information contained in unlabeled data, i.e. facial expressions in the source or target space without corresponding poses. Finally, we present a novel realtime physics-based animation technique allowing to simulate a large range of deformable materials such as fat, flesh, hair, or muscles. This approach could be used to produce more lifelike animations by enhancing the animated avatars with secondary effects. We believe that the realtime face tracking and animation pipeline presented in this thesis has the potential to inspire numerous future research in the area of computer-generated animation. Already, several ideas presented in thesis have been successfully used in industry and this work gave birth to the startup company faceshift AG.Item Sampling and Variance Analysis for Monte Carlo Integration in Spherical Domain(HAL, 2015-09) Singh, GurpritThis dissertation introduces a theoretical framework to study different sampling patterns in the spherical domain and their effects in the evaluation of global illumination integrals. Evaluating illumination (light transport) is one of the most essential aspect in image synthesis to achieve realism which involves solving multi-dimensional space integrals. Monte Carlo based numerical integration schemes are heavily employed to solve these high dimensional integrals. One of the most important aspect of any numerical integration method is sampling. The way samples are distributed on an integration domain can greatly affect the final result. For example, in images, the effects of various sampling patterns appears in the form of either structural artifacts or completely unstructured noise. In many cases, we may get completely false (biased) results due to the sampling pattern used in integration. The distribution of sampling patterns can be characterized using their Fourier power spectra. It is also possible to use the Fourier power spectrum as input, to generate the corresponding sample distribution. This further allows spectral control over the sample distributions. Since this spectral control allows tailoring new sampling patterns directly from the input Fourier power spectrum, it can be used to improve error in integration. However, a direct relation between the error in Monte Carlo integration and the sampling power spectrum is missing. In this work, we propose a variance formulation, that establishes a direct link between the variance in Monte Carlo integration and the power spectra of both the sampling pattern and the integrand involved. To derive our closed-form variance formulation, we use the notion of homogeneous sample distributions that allows expression of error in Monte Carlo integration, only in the form of variance. Based on our variance formulation, we develop an analysis tool that can be used to derive theoretical variance convergence rates of various state-of-the-art sampling patterns. Our analysis give insights to design principles that can be used to tailor new sampling patterns based on the integrand.Item Algorithms for User-Guided Surface Mappings(ETH Zurich, 2015) Diamanti, OlgaComputing mappings between spaces is a very general problem that appears in various forms in geometry processing. They can be used to provide descriptions or representations of shapes, or place shapes in correspondence. Their applications range from surface modeling and analysis to shape matching, morphing, attribute transfer and deformation. This thesis addresses two particular mapping problems that are of interest in the field, namely inter-surface maps and parameterizations. We focus on methods that are suitable for user-guided applications – we do not consider automatic methods, that do not leave space for the user to control the result. Existing meth- ods for the particular sub-problems that we are studying often either suffer from performance limitations, or cannot guarantee that the produced results align with the user’s intent; we improve upon the state of the art in both those respects. The first problem we study in this thesis is that of inter-surface mapping, with given sparse landmark point correspondences. We found that an efficient solution to this otherwise difficult topic emerges if one reformulates the mapping problem as a problem of finding affine combinations of points on the involved shapes. We extend the notion of standard Euclidean weighted averaging to 3D manifold shapes, and introduce a fast approximation that can be used to solve this problem much faster than the state of the art. We showcase applications of this approach in interactive attribute transfer between shapes. Next, we move on to the problem of surface parameterization. Here, we study the problem from the application point of view of surface remeshing; a popular way to generate a quadrilateral mesh for a given triangular mesh is to first compute a global parameterization, which is guided by a tangent vector field. This field then determines the directions of the quadrilateral edges on the output mesh. In order to design such a direction field, recent methods to tackle the problem are based on integer optimization problems, which often suffer from slow performance and local minima. We reformulate the problem in a way that the field design problem becomes a linear problem. We also add more flexibility by allowing for non- orthogonal directions. Still on the same problem of field-aligned surface parameterizations, we notice that the standard way of producing fields –namely, an optimization only focused iiion field smoothness– does not necessarily guarantee that the resulting quadrilateral meshing will be what the user intended in terms of edge directions. This is due to errors introduced in the post-processing of the field, during the later stages of the remeshing pipeline. This renders such fields suboptimal for user-guided meshing applications. We extend our efficient reformulation of the field design problem to generate fields that are guaranteed to not introduce such further errors, and thus make sure that the users obtain the expected results. Additionally, we allow users more flexible control, by supporting assignment of partial constraints for only some of the directions.Item 3D Reconstruction and Rendering from High Resolution Light Fields(ETH Zurich, 2015) Kim, ChangilThis thesis presents a complete processing pipeline of densely sampled, high resolution light fields, from acquisition to rendering. The key components of the pipeline include 3D scene reconstruction, geometry-driven sampling analysis, and controllable multiscopic 3D rendering. The thesis first addresses 3D geometry reconstruction from light fields. We show that dense sampling of a scene attained in light fields allows for more robust and accurate depth estimation without resorting to patch matching and costly global optimization processes. Our algorithm estimates the depth for each and every light ray in the light field with great accuracy, and its pixel-wise depth computation results in particularly favorable quality around depth discontinuities. In fact, most operations are kept localized over small portions of the light field, which by itself is crucial to scalability for higher resolution input and also well suited for efficient parallelized implementations. Resulting reconstructions retain fine details of the scene and exhibit precise localization of object boundaries. While it is the key to the success of our reconstruction algorithm, the dense sampling of light fields entails difficulties when it comes to the acquisition and processing of light fields. This raises a question of optimal sampling density required for faithful geometry reconstruction. Existing works focus more on the alias-free rendering of light fields, and geometry-driven analysis has seen much less research effort. We propose an analysis model for determining sampling locations that are optimal in the sense of high quality geometry reconstruction. This is achieved by analyzing the visibility of scene points and the resolvability of depth and estimating the distribution of reliable estimates over potential sampling locations. A light field with accurate depth information enables an entirely new approach to flexible and controllable 3D rendering. We develop a novel algorithm for multiscopic rendering of light fields which provides great controllability over the perceived depth conveyed in the output. The algorithm synthesizes a pair of stereoscopic images directly from light fields and allows us to control stereoscopic and artistic constraints on a per-pixel basis. It computes non-planar 2D cuts over a light field volume that best meet described constraints by minimizing an energy functional. The output images are synthesized by sampling light rays on the cut surfaces. The algorithm generalizes for multiscopic 3D displays by computing multiple cuts. The resulting algorithms are highly relevant to many application scenarios. It can readily be applied to 3D scene reconstruction and object scanning, depth-assisted segmentation, image-based rendering, and stereoscopic content creation and post-processing, and can also be used to improve the quality of light field rendering that requires depth information such as super-resolution and extended depth of field.Item Sampled and Prefiltered Anti-Aliasing on Parallel Hardware(2015-05) Thomas, AuzingerA fundamental task in computer graphics is the generation of two-dimensional images. Prominent examples are the conversion of text or three-dimensional scenes to formats that can be presented on a raster display. Such a conversion process—often referred to as rasterization or sampling—underlies inherent limitations due to the nature of the output format. This causes not only a loss of information in the rasterization result, which manifests as reduced image sharpness, but also causes corruption of the retained information in form of aliasing artifacts. Commonly observed examples in the final image are staircase artifacts along object silhouettes or Moiré-like patterns. The main focus of this thesis is on the effective removal of such artifacts—a process that is generally referred to as anti-aliasing. This is achieved by removing the offending input information in a filtering step during rasterization. In this thesis, we present different approaches that either minimize computational effort or emphasize output quality. We follow the former objective in the context of an applied scenario from medical visualization. There, we support the investigation of the interiors of blood vessels in complex arrangements by allowing for unrestricted view orientation. Occlusions of overlapping blood vessels are minimized by automatically generating cut-aways with the help of an occlusion cost function. Furthermore, we allow for suitable extensions of the vessel cuts into the surrounding tissue. Utilizing a level of detail approach, these cuts are gradually smoothed with increasing distance from their respective vessels. Since interactive response is a strong requirement for a medical application, we employ fast sample-based anti-aliasing methods in the form of visibility sampling, shading supersampling, and post-process filtering. We then take a step back and develop the theoretical foundations for anti-aliasing methods that follow the second objective of providing the highest degree of output quality. As the main contribution in this context, we present exact anti-aliasing in the form of prefiltering. By computing closed-form solutions of the filter convolution integrals in the continuous domain, we circumvent any issues that are caused by numerical integration and provide mathematically correct results. Together with a parallel hidden-surface elimination, which removes all occluded object parts when rasterizing three-dimensional scenes, we present a ground-truth solution for this setting with exact anti-aliasing. We allow for complex illumination models and perspective-correct shading by combining visibility prefiltering with shading sampling and generate a ground-truth solution for multisampling anti-aliasing. All our aforementioned methods exhibit highly parallel workloads. Throughout the thesis, we present their mapping to massively parallel hardware architectures in the form of graphics processing units. Since our approaches do not map to conventional graphics pipelines, we implement our approach using general-purpose computing concepts. This results in decreased runtime of our methods and makes all of them interactive.Item The Use of Geometric Structures in Graphics and Optimization(2015) Bus, NorbertReal-world data has a large geometric component, exhibiting significant geometric patterns. Therefore exploiting the geometric nature of data to design efficient methods has became a very important topic in several scientific fields, e.g., computational geometry, discrete geometry, computer graphics, computer vision. In this thesis we use geometric structures to design efficient algorithms for problems in two domains, computer graphics and combinatorial optimization. Part I focuses on a geometric data structure called well-separated pair decomposition and its usage for one of the most challenging problems in computer graphics, namely efficient photo-realistic rendering. One solution is the family of many-lights methods that approximate global illumination by individually computing illumination from a large number of virtual point lights (VPLs) placed on surfaces. Considering each VPL individually results in a vast number of calculations. One successful strategy to reduce computations is to group the VPLs into a small number of clusters that are treated as individual lights with respect to each point to be shaded. We use the well-separated pair decomposition of points as a basis for a data structure for pre-computing and compactly storing a set of view independent candidate VPL clusterings showing that a suitable clustering of the VPLs can be efficiently extracted from this data structure. We show that instead of clustering points and/or VPLs independently, what is required is to cluster the product-space of the set of points to be shaded and the set of VPLs based on the induced pairwise illumination. Additionally we propose an adaptive sampling technique to reduce the number of visibility queries for each product-space cluster. Our method handles any light source that can be approximated with virtual point lights (VPLs), highly glossy materials and outperforms previous state-of-the-art methods. Part II focuses on developing new approximation algorithms for a fundamental NP-complete problem in computational geometry. It focuses on the minimum hitting set problem, particularly on the case where given a set of points and a set of disks, we wish to compute the minimum-sized subset of the points that hits all disks. It turns out that efficient algorithms for geometric hitting set rely on a key geometric structure, called eps-net. We give an algorithm that uses only Delaunay triangulations to construct eps-nets of size 13.4/eps and we provide a practical implementation of a technique to calculate hitting sets in near-linear time using small sized eps-nets. Our results yield a 13.4 approximation for the hitting set problem with an algorithm that runs efficiently even on large data sets. For smaller datasets, we present an implementation of the local search technique along with tight approximation bounds for its approximation factor, yielding an (8 + eps)-approximation algorithm with running time O(n^2.34). Our work related to fundamental computational geometry problems also includes a novel dynamic convex hull algorithm for simple polygonal chains handling insertion or deletion of a point in amortized constant time.Item A Framework for Local Terrain Deformation Based on Diffusion Theory(University of Sussex, 2015-10) Gilardi, MarcoTerrains have a key role in making outdoor virtual scenes believable and immersive as they form the support for every other natural element in the scene. Although important, terrains are often given limited interactivity in real-time applications. However, in nature, terrains are dynamic and interact with the rest of the environment changing shape on different levels, from tracks left by a person running on a gravel soil (micro-scale), to avalanches on the side of a mountain (macro-scale). The challenge in representing dynamic terrains correctly is that the soil that forms them is vastly heterogeneous and behaves differently depending on its composition. This heterogeneity introduces difficulties at different levels in dynamic terrains simulations, from modelling the large amount of different elements that compose the oil to simulating their dynamic behaviour. This work presents a novel framework to simulate multi-material dynamic terrains by taking into account the soil composition and its heterogeneity. In the proposed framework soil information is obtained from a material description map applied to the terrain mesh. This information is used to compute deformations in the area of interaction using a novel mathematical model based on diffusion theory. The deformations are applied to the terrain mesh in different ways depending on the distance of the area of interaction from the camera and the soil material. Deformations away from the camera are simulated by dynamically displacing normals. While deformations in a neighbourhood of the camera are represented by displacing the terrain mesh, which is locally tessellated to better fit the displacement. For gravel based soils the terrain details are added near the camera by reconstructing the meshes of the small rocks from the texture image, thus simulating both micro and macro-structure of the terrain. The outcome of the framework is a realistic interactive dynamic terrain animation in real-time.Item Scene Reconstruction from Multi-Scale Input Data(TU Darmstadt ULB, 2015-06-18) Fuhrmann, SimonGeometry acquisition of real-world objects by means of 3D scanning or stereo reconstruction constitutes a very important and challenging problem in computer vision. 3D scanners and stereo algorithms usually provide geometry from one viewpoint only, and several of the these scans need to be merged into one consistent representation. Scanner data generally has lower noise levels than stereo methods and the scanning scenario is more controlled. In image-based stereo approaches, the aim is to reconstruct the 3D surface of an object solely from multiple photos of the object. In many cases, the stereo geometry is contaminated with noise and outliers, and exhibits large variations in scale. Approaches that fuse such data into one consistent surface must be resilient to such imperfections. In this thesis, we take a closer look at geometry reconstruction using both scanner data and the more challenging image-based scene reconstruction approaches. In particular, this work focuses on the uncontrolled setting where the input images are not constrained, may be taken with different camera models, under different lighting and weather conditions, and from vastly different points of view. A typical dataset contains many views that observe the scene from an overview perspective, and relatively few views capture small details of the geometry. What results from these datasets are surface samples of the scene with vastly different resolution. As we will show in this thesis, the multi-resolution, or, "multi-scale" nature of the input is a relevant aspect for surface reconstruction, which has rarely been considered in literature yet. Integrating scale as additional information in the reconstruction process can make a substantial difference in surface quality. We develop and study two different approaches for surface reconstruction that are able to cope with the challenges resulting from uncontrolled images. The first approach implements surface reconstruction by fusion of depth maps using a multi-scale hierarchical signed distance function. The hierarchical representation allows fusion of multi-resolution depth maps without mixing geometric information at incompatible scales, which preserves detail in high-resolution regions. An incomplete octree is constructed by incrementally adding triangulated depth maps to the hierarchy, which leads to scattered samples of the multi-resolution signed distance function. A continuous representation of the scattered data is defined by constructing a tetrahedral complex, and a final, highly-adaptive surface is extracted by applying the Marching Tetrahedra algorithm. A second, point-based approach is based on a more abstract, multi-scale implicit function defined as a sum of basis functions. Each input sample contributes a single basis function which is parameterized solely by the sample's attributes, effectively yielding a parameter-free method. Because the scale of each sample controls the size of the basis function, the method automatically adapts to data redundancy for noise reduction and is highly resilient to the quality-degrading effects of low-resolution samples, thus favoring high-resolution surfaces. Furthermore, we present a robust, image-based reconstruction system for surface modeling: MVE, the Multi-View Environment. The implementation provides all steps involved in the pipeline: Calibration and registration of the input images, dense geometry reconstruction by means of stereo, a surface reconstruction step and post-processing, such as remeshing and texturing. In contrast to other software solutions for image-based reconstruction, MVE handles large, uncontrolled, multi-scale datasets as well as input from more controlled capture scenarios. The reason lies in the particular choice of the multi-view stereo and surface reconstruction algorithms. The resulting surfaces are represented using a triangular mesh, which is a piecewise linear approximation to the real surface. The individual triangles are often so small that they barely contribute any geometric information and can be ill-shaped, which can cause numerical problems. A surface remeshing approach is introduced which changes the surface discretization such that more favorable triangles are created. It distributes the vertices of the mesh according to a density function, which is derived from the curvature of the geometry. Such a mesh is better suited for further processing and has reduced storage requirements. We thoroughly compare the developed methods against the state-of-the art and also perform a qualitative evaluation of the two surface reconstruction methods on a wide range of datasets with different properties. The usefulness of the remeshing approach is demonstrated on both scanner and multi-view stereo data.Item Interactive Integrated Exploration and Management of Visualization Parameters(TU Wien, 2015-07) Mindek, PeterVisualization algorithms are parameterized to offer universality in terms of handling various data types, showing different aspects of the visualized data, or producing results useful for domain experts from different fields. Hence, input parameters are an important aspect of the visualization process. Their exploration and management are tasks which enable the visualization reusability, portability, and interdisciplinary communication. With increasing availability of visualization systems, which are suitable for a great variety of tasks, their complexity increases as well. This usually involves many input parameters necessary for the meaningful visualization of data. Multiple input parameters form parameter spaces which are too large to be explored by brute-force. Knowing the properties of a parameter space is often beneficial for improving data visualization. Therefore, it is important for domain experts utilizing data visualization to have tools for automatic parameter specification and for aiding the manual parameter setting. In this thesis, we review existing approaches for parameter-space visualization, exploration, and management. These approaches are used with a great variety of underlying algorithms. We focus on their applicability to visualization algorithms. We propose three methods solving specific problems arising from the fact that the output of a visualization algorithm is an image, which is challenging to process automatically and often needs to be analyzed by a human. First, we propose a method for the exploration of parameter-spaces of visualization algorithms. The method is used to understand effects of combinations of parameters and parts of the internal structure of the visualization algorithms on the final image result. The exploration is carried out by specifying semantics for localized parts of the visualization images in the form of positive and negative examples influenced by a set of input parameters or parts of the visualization algorithm itself. After specifying the localized semantics, global effects of the specified components of the visualization algorithm can be observed. The method itself is independent from the underlying algorithm. Subsequently, we present a method for managing image-space selections in visualizations and automatically link them with the context in which they were created. The context is described by the values of the visualization parameters influencing the output image. The method contains a mechanism for linking additional views to the selections, allowing the user an effective management of the visualization parameters whose effects are localized to certain areas of the visualizations. We present various applications for the method, as well as an implementation in the form of a library, which is ready to be used in existing visualization systems. Our third method is designed to integrate dynamic parameters stored during a multiplayer video game session by the individual participating players. For each player, the changing parameter values of the game describe their view of the gameplay. Integrating these multiple views into a single continuous visual narrative provides means for effective summarization of gameplays, useful for entertainment, or even gameplay analysis purposes by semi-professional or professional players. We demonstrate the utility of our approach on an existing video game by producing a gameplay summary of a multiplayer game session. The proposed method opens possibilities for further research in the areas of storytelling, or at a more abstract level, parameter integration for visual computing algorithms.Item Scheduling activities under spatial and temporal constraints to populate virtual urban environments(Université Rennes 1, 2015-07-17) Jorgensen, Carl-JohanCrowd simulation models usually aim at producing visually credible crowds with the intent of giving life to virtual environments. Our work focusses on generating statistically consistent behaviours that can be used to pilot crowd simulation models over long periods of time, up to multiple days. In real crowds, people's behaviours mainly depend on the activities they intend to perform. The way this activity is scheduled rely on the close interaction between the environment, space and time constraints associated with the activity and personal characteristics of individuals. Compared to the state of the art, our model better handle this interaction. Our main contributions lie in the domain of activity scheduling and path planning. First, we propose an individual activity scheduling process and its extension to cooperative activity scheduling. Based on descriptions of the environment, of intended activities and of agents' characteristics, these processes generate a task schedule for each agent. Locations where the tasks should be performed are selected and a relaxed agenda is produced. This task schedule is compatible with spatial and temporal constraints associated with the environment and with the intended activity of the agent and of other cooperating agents. It also takes into account the agents personal characteristics, inducing diversity in produced schedules. We show that our model produces schedules statistically coherent with the ones produced by humans in the same situations. Second, we propose a hierarchical path-planning process. It relies on an automatic environment analysis process that produces a semantically coherent hierarchical representation of virtual cities. The hierarchical nature of this representation is used to model different levels of decision making related to path planning. A coarse path is first computed, then refined during navigation when relevant information is available. It enable the agent to seamlessly adapt its path to unexpected events. The proposed model handles long term rational decisions driving the navigation of agents in virtual cities. It considers the strong relationship between time, space and activity to produce more credible agents' behaviours. It can be used to easily populate virtual cities in which observable crowd phenomena emerge from individual activities.Item Reinforcement learning in a Multi-agent Framework for Pedestrian Simulation(ProQuest, 2014-10-23) Martinez-Gil, FranciscoThis thesis proposes a new approach to pedestrian simulation based on machine learning techniques. Specifically, the work proposes the use of reinforcement learning techniques to build a decision-making modulefor pedestrian navigation. The thesis presents a multi-agent framework in which each agent is an embodied 3D agent calibrated with human features. The virtual worls is also a 3D world in which objects such as walls or doors are placed. The agents perceive their local neighborhood (objects and the rest of agents) and learn to move in this virtual world towards a place inside the environment. The thesis studies different algorithmic approaches based on reinforcement learning and analyzes the results in different scenarios. These scenarios are classic studied situations in the field of pedestrian modelling and simulation (bottlenecks, crossings inside a narrow corridor,...). The results show that the approach is capable of solving successfully the navigational problems. Besides emergent collective behaviors appear such as arch-like grouping around an exit in the bottleneck problem or lanes formation in the crossing inside a corridor scenario. The work opens a new research line in the pedestrian simulation studies which offers advantages as: - The behavioral design is in charge of the learning process and it is not coded by humans. - The agents learn independently different behaviors attending to thheir personal experiencies and interactions with the 3D world. - The learned decision-making module is computationally efficient (because the learned behavior is stored in form of a table or a linear function approximator). The approach has also limitations: - The learned behaviors can not be edited directly, making non trivial the task of implementing authoring tools. - The quality of the learned behaviors is not homogeneous. There are agents that learn very well their task but others do not. -The learned process is not controllable in terms of when and whatis learned in each moment.Item Marker-less Motion Capture in General Scenes with Sparse Multi-camera Setups(2015-12-09) Elhayek, AhmedHuman motion-capture from videos is one of the fundamental problems in computer vision and computer graphics. Its applications can be found in a wide range of industries. Even with all the developments in the past years, industry and academia alike still rely on complex and expensive marker-based systems. Many state-of-the-art marker-less motioncapture methods come close to the performance of marker-based algorithms, but only when recording in highly controlled studio environments with exactly synchronized, static and su ciently many cameras. While relative to marker-based systems, this yields an easier apparatus with a reduced setup time, the hurdles towards practical application are still large and the costs are considerable. By being constrained to a controlled studio, marker-less methods fail to fully play out their advantage of being able to capture scenes without actively modifying them. In the area of marker-less human motion-capture, this thesis proposes several novel algorithms for simplifying the motion-capture to be applicable in new general outdoor scenes. The rst is an optical multi-video synchronization method which achieves subframe accuracy in general scenes. In this step, the synchronization parameters of multiple videos are estimated. Then, we propose a spatio-temporal motion-capture method which uses the synchronization parameters for accurate motion-capture with unsynchronized cameras. Afterwards, we propose a motion capture method that works with moving cameras, where multiple people are tracked even in front of cluttered and dynamic backgrounds with potentially moving cameras. Finally, we reduce the number of cameras employed by proposing a novel motion-capture method which uses as few as two cameras to capture high-quality motion in general environments, even outdoors. The methods proposed in this thesis can be adopted in many practical applications to achieve similar performance as complex motion-capture studios with a few consumer-grade cameras, such as mobile phones or GoPros, even for uncontrolled outdoor scenes.Item Pattern Search for the Visualization of Scalar, Vector, and Line Fields(2015-12-18) Wang, ZhongjieThe main topic of this thesis is pattern search in data sets for the purpose of visual data analysis. By giving a reference pattern, pattern search aims to discover similar occurrences in a data set with invariance to translation, rotation and scaling. To address this problem, we developed algorithms dealing with different types of data: scalar fields, vector fields, and line fields. For scalar fields, we use the SIFT algorithm (Scale-Invariant Feature Transform) to find a sparse sampling of prominent features in the data with invariance to translation, rotation, and scaling. Then, the user can define a pattern as a set of SIFT features by e.g. brushing a region of interest. Finally, we locate and rank matching patterns in the entire data set. Due to the sparsity and accuracy of SIFT features, we achieve fast and memory-saving pattern query in large scale scalar fields. For vector fields, we propose a hashing strategy in scale space to accelerate the convolution-based pattern query. We encode the local flow behavior in scale space using a sequence of hierarchical base descriptors, which are pre-computed and hashed into a number of hash tables. This ensures a fast fetching of similar occurrences in the flow and requires only a constant number of table lookups. For line fields, we present a stream line segmentation algorithm to split long stream lines into globally-consistent segments, which provides similar segmentations for similar flow structures. It gives the benefit of isolating a pattern from long and dense stream lines, so that our patterns can be defined sparsely and have a significant extent, i.e., they are integration-based and not local. This allows for a greater flexibility in defining features of interest. For user-defined patterns of curve segments, our algorithm finds similar ones that are invariant to similarity transformations. Additionally, we present a method for shape recovery from multiple views. This semi-automatic method fits a template mesh to high-resolution normal data. In contrast to existing 3D reconstruction approaches, we accelerate the data acquisition time by omitting the structured light scanning step of obtaining low frequency 3D information.Item Exploratory Search in Time-Oriented Primary Data(2015-11-20) Bernard, JürgenIn a variety of research fields, primary data that describes scientific phenomena in an original condition is obtained. Time-oriented primary data, in particular, is an indispensable data type, derived from complex measurements depending on time. Today, time-oriented primary data is collected at rates that exceed the domain experts’ abilities to seek valuable information undiscovered in the data. It is widely accepted that the magnitudes of uninvestigated data will disclose tremendous knowledge in data-driven research, provided that domain experts are able to gain insight into the data. Domain experts involved in data-driven research urgently require analytical capabilities. In scientific practice, predominant activities are the generation and validation of hypotheses. In analytical terms, these activities are often expressed in confirmatory and exploratory data analysis. Ideally, analytical support would combine the strengths of both types of activities. Exploratory search (ES) is a concept that seamlessly includes information-seeking behaviors ranging from search to exploration. ES supports domain experts in both gaining an understanding of huge and potentially unknown data collections and the drill-down to relevant subsets, e.g., to validate hypotheses. As such, ES combines predominant tasks of domain experts applied to data-driven research. For the design of useful and usable ES systems (ESS), data scientists have to incorporate different sources of knowledge and technology. Of particular importance is the state-of-the-art in interactive data visualization and data analysis. Research in these factors is at heart of Information Visualization (IV) and Visual Analytics (VA). Approaches in IV and VA provide meaningful visualization and interaction designs, allowing domain experts to perform the information-seeking process in an effective and efficient way. Today, bestpractice ESS almost exclusively exist for textual data content, e.g., put into practice in digital libraries to facilitate the reuse of digital documents. For time-oriented primary data, ES mainly remains at a theoretical state.Item Printing Beyond Color: Spectral and Specular Reproduction(2015-11-30) Samadzadegan, SepidehFor accurate printing (reproduction), two important appearance attributes to consider are color and gloss. These attributes are related to two topics focused on in this dissertation: spectral reproduction and specular (gloss) printing. In the conventional printing workflow known as the metameric printing workflow, which we use mostly nowadays, high-quality prints -- in terms of colorimetric accuracy -- can be achieved only under a predefined illuminant (i.e. an illuminant that the printing pipeline is adjusted to; e.g. daylight). While this printing workflow is useful and sufficient for many everyday purposes, in some special cases, such as artwork (e.g. painting) reproduction, security printing, accurate industrial color communication and so on, in which accurate reproduction of an original image under a variety of illumination conditions (e.g. daylight, tungsten light, museum light, etc.) is required, metameric reproduction may produce satisfactory results only with luck. Therefore, in these cases, another printing workflow, known as spectral printing pipeline must be used, with the ideal aim of illuminant-invariant match between the original image and the reproduction. In this workflow, the reproduction of spectral raw data (i.e. reflectances in the visible wavelength range), rather than reproduction of colorimetric values (colors) alone (under a predefined illuminant) is taken into account. Due to the limitations of printing systems extant, the reproduction of all reflectances is not possible even with multi-channel (multi-colorant) printers. Therefore, practical strategies are required in order to map non-reproducible reflectances into reproducible spectra and to choose appropriate combinations of printer colorants for the reproduction of the mapped reflectances. For this purpose, an approach called Spatio-Spectral Gamut Mapping and Separation, SSGMS, was proposed, which results in almost artifact-free spectral reproduction under a set of various illuminants. The quality control stage is usually the last stage in any printing pipeline. Nowadays, the quality of the printout is usually controlled only in terms of colorimetric accuracy and common printing artifacts. However, some gloss-related artifacts, such as gloss-differential (inconsistent gloss appearance across an image, caused mostly by variations in deposited ink area coverage on different spots), are ignored, because no strategy to avoid them exists. In order to avoid such gloss-related artifacts and to control the glossiness of the printout locally, three printing strategies were proposed. In general, for perceptually accurate reproduction of color and gloss appearance attributes, understanding the relationship between measured values and perceived magnitudes of these attributes is essential. There has been much research into reproduction of colors within perceptually meaningful color spaces, but little research from the gloss perspective has been carried out. Most of these studies are based on simulated display-based images (mostly with neutral colors) and do not take real objects into account. In this dissertation, three psychophysical experiments were conducted in order to investigate the relationship between measured gloss values (objective quantities) and perceived gloss magnitudes (subjective quantities) using real colored samples printed by the aforementioned proposed printing strategies. These experiments revealed that the relationship mentioned can be explained by a Power function according to Stevens' Power Law, considering almost the entire gloss range. Another psychophysical experiment was also conducted in order to investigate the interrelation between perceived surface gloss and texture, using 2.5D samples printed in two different texture types and with various gloss levels and texture elevations. According to the results of this experiment, different macroscopic texture types and levels (in terms of texture elevation) were found to influence the perceived surface gloss level slightly. No noticeable influence of surface gloss on the perceived texture level was observed, indicating texture constancy regardless of the gloss level printed. The SSGMS approach proposed for the spectral reproduction, the three printing strategies presented for gloss printing, and the results of the psychophysical experiments conducted on gloss printing and appearance can be used to improve the overall print quality in terms of color and gloss reproduction.Item Interactive Physically Based Simulation — Efficient Higher-Order Elements, Multigrid Approaches and Massively Parallel Data Structures(2015-12-07) Weber, DanielThis thesis covers interactive physically based simulation for applications such as computer games or virtual environments. Interactivity, i.e., the option that a user can influence a system, imposes challenging requirements on the simulation algorithms. A simple way to achieve this goal is to drastically limit the resolution in order to guarantee this low computation time. However, with current methods the number of degrees of freedom will be rather low, which results in a low degree of realism. This is due to the fact that not every detail that is important for realistically representing the physical system can be resolved. This thesis contributes to interactive physically based simulation by developing novel methods and data structures. These can be associated with the three pillars of this thesis: more accurate discrete representations, efficient methods for linear systems, and data structures and methods for massively parallel computing. The novel approaches are evaluated in two application areas relevant in computer generated animation: simulation of dynamic volumetric deformation and fluid dynamics. The resulting accelerations allow for a higher degree of realism because the number of elements or the resolution can be significantly increased.Item Physically based rendering of synthetic objects in real environments(Linköping University Electronic Press, 2016-12-04) Joel, KronanderThis thesis presents methods for photorealistic rendering of virtual objects so that they can be seamlessly composited into images of the real world. To generate predictable and consistent results, we study physically based methods, which simulate how light propagates in a mathematical model of the augmented scene. This computationally challenging problem demands both efficient and accurate simulation of the light transport in the scene, as well as detailed modeling of the geometries, illumination conditions, and material properties. In this thesis, we discuss and formulate the challenges inherent in these steps and present several methods to make the process more efficient. In particular, the material contained in this thesis addresses four closely related areas: HDR imaging, IBL, reflectance modeling, and efficient rendering. The thesis presents a new, statistically motivated algorithm for HDR reconstruction from raw camera data combining demosaicing, denoising, and HDR fusion in a single processing operation. The thesis also presents practical and robust methods for rendering with spatially and temporally varying illumination conditions captured using omnidirectional HDR video. Furthermore, two new parametric BRDF models are proposed for surfaces exhibiting wide angle gloss. Finally, the thesis also presents a physically based light transport algorithm based on Markov Chain Monte Carlo methods that allows approximations to be used in place of exact quantities, while still converging to the exact result. As illustrated in the thesis, the proposed algorithm enables efficient rendering of scenes with glossy transfer and heterogenous participating media.Item Geometric Methods for Realistic Animation of Faces(ETH Zurich, 2015) Bermano, Amit HaimRealistic facial synthesis is one of the most fundamental problems in computer graphics, and has been sought after for approximately four decades. It is desired in a wide variety of fields, such as character animation for films and advertising, computer games, video teleconferencing, user-interface agents and avatars, and facial surgery planning. Humans, on the other hand, are experts in identifying every detail and every regularity or variation in proportion from one individual to the next. The task of creating a realistic human face is elusive due to this, as well as many other factors. Among which are complex surface details, spatially and temporally varying skin texture and subtle emotions that are conveyed through even more subtle motions. In this thesis, we present the most commonly practiced facial content creation process, and contribute to the quality of each of its steps. The proposed algorithms significantly increase the level of realism attained by each step and therefore substantially reduce the amount of manual labor required for production quality facial content. The thesis contains three parts, each contributing to one step of the facial content creation pipeline. In the first part, we aim at greatly increasing the fidelity of facial performance captures, and present the first method for detailed spatio-temporal reconstruction of eyelids. Easily integrable with existing high quality facial performance capture approaches, this method generates a person-specific, time-varying eyelid reconstruction with anatomically plausible deformations. Our approach is to combine a geometric deformation model with image data, leveraging multi-view stereo, optical flow, contour tracking and wrinkle detection from local skin appearance. Our deformation model serves as a prior that enables reconstruction of eyelids even under strong self-occlusions caused by rolling and folding skin as the eye opens and closes. In the second part, we contribute to the authoring step of the creation process. We present a method for adding fine-scale details and expressiveness to lowresolution art-directed facial performances. Employing a high-resolution facial performance capture system, we augment artist friendly content, such as those created manually using a rig, via marker-based capture, by fitting a morphable model to a video, or through Kinect-based reconstruction. From the high fidelity captured data, our system encodes subtle spatial and temporal deformation details specific to that particular individual, and composes the relevant ones to the desired input animation. The resulting animations exhibit compelling animations with nuances and fine spatial details that match captured performances, while preserving the artistic intent authored by the low-resolution input sequences, outperforming current state-of-the-art in example-based facial animation. The third part of the dissertation proposes to enrich digital facial content by adding a significant sense of presence. Replacing the classic 2D or 3D displaying techniques of digital content, we propose the first complete process for augmenting deforming physical avatars using projector-based illumination. Physical avatars have been long used to give physical presence to a character, both in the field of entertainment and teleconferencing. Using a human-shaped display surface provides depth cues and multiple observers with their own perspectives. Such physical avatars, however, suffer from limited movement and expressiveness due to mechanical constraints. Given an input animation, our system decomposes the motion into low-frequency motion that can be physically reproduced by a robotic head and high-frequency details that are added using projected shading. The result of our system is a highly expressive physical avatar that features facial details and motion otherwise unattainable due to physical constraints.Item Plenoptic Light Transport(2015-11-27) Jarabo, AdriánIn this thesis we focus on the multidimensional nature of light transport as described by the plenoptic function, and in particular in the angular and temporal domains. While traditional imaging has been limited to bidimensional images, the emerging field of Computational Imaging has made increasingly available more complex multidimensional visual data, disambiguating additional domains of the plenoptic function. However, this higher dimensionality requires changing the way that visual information is processed, manipulated, visualized or synthesized. In this thesis we present contributions on these topics, addressing the challenges of adapting and rethinking them to handle higher-dimensional visual information. Specifically, within the angular domain we focus on light field editing, studying interaction paradigms and user workflows when interacting with light fields, and on spatio-angular filtering of complex appearances modeled with BTFs, studying how filtering affects appearance perception. On the other hand, in the temporal domain we focus on transient light transport, where the speed of light cannot longer be considered infinite, including contributions on capture and data processing, light transport simulation and visualization of time-resolved data.