Browsing by Author "Mitra, Niloy"

Now showing 1 - 4 of 4
  • Thumbnail Image
    Item
    Deep Learning for Computer Graphics and Geometry Processing
    (The Eurographics Association, 2019) Bronstein, Michael; Guibas, Leonidas; Kokkinos, Iasonas; Litany, Or; Mitra, Niloy; Monti, Federico; Rodolà, Emanuele; Jakob, Wenzel and Puppo, Enrico
    In computer graphics and geometry processing, many traditional problems are now becoming increasingly handled by data-driven methods. In an increasing variety of problem settings, deep networks are state-of-the-art, beating dedicated hand-crafted methods by significant margins. This tutorial gives an organized overview of core theory, practice, and graphics-related applications of deep learning.
  • No Thumbnail Available
    Item
    Discovering Structured Variations Via Template Matching
    (© 2017 The Eurographics Association and John Wiley & Sons Ltd., 2017) Ceylan, Duygu; Dang, Minh; Mitra, Niloy J.; Neubert, Boris; Pauly, Mark; Chen, Min and Zhang, Hao (Richard)
    Understanding patterns of variation from raw measurement data remains a central goal of shape analysis. Such an understanding reveals which elements are repeated, or how elements can be derived as structured variations from a common base element. We investigate this problem in the context of 3D acquisitions of buildings. Utilizing a set of template models, we discover geometric similarities across a set of building elements. Each template is equipped with a deformation model that defines variations of a base geometry. Central to our algorithm is a simultaneous template matching and deformation analysis that detects patterns across building elements by extracting similarities in the deformation modes of their matching templates. We demonstrate that such an analysis can successfully detect structured variations even for noisy and incomplete data. Understanding patterns of variation from raw measurement data remains a central goal of shape analysis. Such an understanding reveals which elements are repeated, or how elements can be derived as structured variations from a common base element. We investigate this problem in the context of 3D acquisitions of buildings. Utilizing a set of template models, we discover geometric similarities across a set of building elements. Each template is equipped with a deformation model that defines variations of a base geometry.
  • Thumbnail Image
    Item
    POP: Full Parametric model Estimation for Occluded People
    (The Eurographics Association, 2019) Marin, Riccardo; Melzi, Simone; Mitra, Niloy J.; Castellani, Umberto; Biasotti, Silvia and Lavoué, Guillaume and Veltkamp, Remco
    In the last decades, we have witnessed advances in both hardware and associated algorithms resulting in unprecedented access to volumes of 2D and, more recently, 3D data capturing human movement. We are no longer satisfied with recovering human pose as an image-space 2D skeleton, but seek to obtain a full 3D human body representation. The main challenges in acquiring 3D human shape from such raw measurements are identifying which parts of the data relate to body measurements and recovering from partial observations, often arising out of severe occlusion. For example, a person occluded by a piece of furniture, or being self-occluded in a profile view. In this paper, we propose POP, a novel and efficient paradigm for estimation and completion of human shape to produce a full parametric 3D model directly from single RGBD images, even under severe occlusion. At the heart of our method is a novel human body pose retrieval formulation that explicitly models and handles occlusion. The retrieved result is then refined by a robust optimization to yield a full representation of the human shape. We demonstrate our method on a range of challenging real world scenarios and produce high-quality results not possible by competing alternatives. The method opens up exciting AR/VR application possibilities by working on 'in-the-wild' measurements of human motion.
  • Thumbnail Image
    Item
    Walk2Map: Extracting Floor Plans from Indoor Walk Trajectories
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Mura, Claudio; Pajarola, Renato; Schindler, Konrad; Mitra, Niloy; Mitra, Niloy and Viola, Ivan
    Recent years have seen a proliferation of new digital products for the efficient management of indoor spaces, with important applications like emergency management, virtual property showcasing and interior design. While highly innovative and effective, these products rely on accurate 3D models of the environments considered, including information on both architectural and non-permanent elements. These models must be created from measured data such as RGB-D images or 3D point clouds, whose capture and consolidation involves lengthy data workflows. This strongly limits the rate at which 3D models can be produced, preventing the adoption of many digital services for indoor space management. We provide a radical alternative to such data-intensive procedures by presentingWalk2Map, a data-driven approach to generate floor plans only from trajectories of a person walking inside the rooms. Thanks to recent advances in data-driven inertial odometry, such minimalistic input data can be acquired from the IMU readings of consumer-level smartphones, which allows for an effortless and scalable mapping of real-world indoor spaces. Our work is based on learning the latent relation between an indoor walk trajectory and the information represented in a floor plan: interior space footprint, portals, and furniture. We distinguish between recovering area-related (interior footprint, furniture) and wall-related (doors) information and use two different neural architectures for the two tasks: an image-based Encoder-Decoder and a Graph Convolutional Network, respectively. We train our networks using scanned 3D indoor models and apply them in a cascaded fashion on an indoor walk trajectory at inference time. We perform a qualitative and quantitative evaluation using both trajectories simulated from scanned models of interiors and measured, real-world trajectories, and compare against a baseline method for image-to-image translation. The experiments confirm that our technique is viable and allows recovering reliable floor plans from minimal walk trajectory data.