WICED: Eurographics Workshop on Intelligent Cinematography and Editing

Permanent URI for this community

https://diglib.eg.org/handle/10.2312/12282

Browse

Now showing 1 - 20 of 52

Analysing Cinematography with Embedded Constrained Patterns
(The Eurographics Association, 2016) Wu, Hui-Yin; Christie, Marc; M. Christie and Q. Galvane and A. Jhala and R. Ronfard
Cinematography carries messages on the plot, emotion, or more general feeling of the film. Yet cinematographic devices are often overlooked in existing approaches to film analysis. In this paper, we present Embedded Constrained Patterns (ECPs), a dedicated query language to search annotated film clips for sequences that fulfill complex stylistic constraints. ECPs are groups of framing and sequencing constraints defined using vocabulary in film textbooks. Using a set algorithm, all occurrences of the ECPs can be found in annotated film sequences. We use a film clip from the Lord of the Rings to demonstrate a range of ECPs that can be detected, and analyse them in relation to story and emotions in the film.
Analyzing Elements of Style in Annotated Film Clips
(The Eurographics Association, 2017) Wu, Hui-Yin; Galvane, Quentin; Lino, Christophe; Christie, Marc; William Bares and Vineet Gandhi and Quentin Galvane and Remi Ronfard
This paper presents an open database of annotated film clips together with an analysis of elements of film style related to how the shots are composed, how the transitions are performed between shots and how the shots are sequenced to compose a film unit. The purpose is to initiate a shared repository pertaining to elements of film style which can be used by computer scientists and film analysts alike. Though both research communities rely strongly on the availability of such information to foster their findings, current databases are either limited to low-level features (such as shots lengths, color and luminance information), contain noisy data, or are not available to the communities. The data and analysis we provide open exciting perspectives as to how computational approaches can rely more thoroughly on information and knowledge extracted from existing movies, and also provide a better understanding of how elements of style are arranged to construct a consistent message.
Automated Cinematography with Unmanned Aerial Vehicles
(The Eurographics Association, 2016) Galvane, Quentin; Fleureau, Julien; Tariolle, Francois-Louis; Guillotel, Philippe; M. Christie and Q. Galvane and A. Jhala and R. Ronfard
The rise of Unmanned Aerial Vehicles and their increasing use in the cinema industry calls for the creation of dedicated tools. Though there is a range of techniques to automatically control drones for a variety of applications, none have considered the problem of producing cinematographic camera motion in real-time for shooting purposes. In this paper we present our approach to UAV navigation for autonomous cinematography. The contributions of this research are twofold: (i) we adapt virtual camera control techniques to UAV navigation; (ii) we introduce a drone-independent platform for high-level user interactions that integrates cinematographic knowledge. The results presented in this paper demonstrate the capacities of our tool to capture live movie scenes involving one or two moving actors.
Automatic Camera Selection and PTZ Canvas Steering for Autonomous Filming of Reality TV
(The Eurographics Association, 2017) Callemein, Timothy; Ranst, Wiebe Van; Goedemé, Toon; William Bares and Vineet Gandhi and Quentin Galvane and Remi Ronfard
Reality TV shows that follow people in their day-to-day lives are not a new concept. However, the traditional methods used in the industry require a lot of manual labor and need the presence of at least one physical camera man. Because of this, the subjects tend to behave differently when they are aware of being recorded. This paper presents an approach to follow people in their day-to-day lives, for long periods of time (months to years), while being as unobtrusive as possible. To do this, we use unmanned cinematographically-aware cameras hidden in people's houses. Our contribution in this paper is twofold: First, we create a system to limit the amount of recorded data by intelligently controlling a video switch matrix, in combination with a multi-channel recorder. Second, we create a virtual camera man by controlling a PTZ camera to automatically make cinematographically pleasing shots. Throughout this paper, we worked closely with a real camera crew, enabling us to compare the results of our system to the work of trained professionals. This work was originally published in MVA 2017, as T. Callemein, W. Van Ranst and T. Goedemé, "The Autonomous hidden Camera Crew".
Automatic Lighting Design from Photographic Rules
(The Eurographics Association, 2016) Wambecke, Jérémy; Vergne, Romain; Bonneau, Georges-Pierre; Thollot, Joëlle; M. Christie and Q. Galvane and A. Jhala and R. Ronfard
Lighting design is crucial in 3D scenes modeling for its ability to provide cues to understand the objects shape. However a lot of time, skills, trials and errors are required to obtain a desired result. Existing automatic lighting methods for conveying the shape of 3D objects are based either on costly optimizations or on non-realistic shading effects. Also they do not take the material information into account. In this paper, we propose a new method that automatically suggests a lighting setup to reveal the shape of a 3D model, taking into account its material and its geometric properties. Our method is independent from the rendering algorithm. It is based on lighting rules extracted from photography books, applied through a fast and simple geometric analysis. We illustrate our algorithm on objects having different shapes and materials, and we show by both visual and metric evaluation that it is comparable to optimization methods in terms of lighting setups quality. Thanks to its genericity our algorithm could be integrated in any rendering pipeline to suggest appropriate lighting.
CaMor: Screw Interpolation between Perspective Projections of Partial Views of Rectangular Images
(The Eurographics Association, 2017) Raghuraman, Gokul; Barrash, Nicholas; Rossignac, Jarek; William Bares and Vineet Gandhi and Quentin Galvane and Remi Ronfard
CaMor is a tool for generating an animation from a single drawing or photograph that represents a partial view of a perspective projection of a planar shape or image that contains portions of only 3 edges of an unknown rectangle. The user identifies these portions and indicates where the corresponding lines should be at the end of the animation. CaMor produces a non-affine animation of the entire plane by combining (1) a new rectification procedure that identifies the orientation in 3D of a rectangle from the partial image of its perspective projection, (2) a depth adjustment that ensures that the two rectified rectangles are congruent in 3D, (3) a screw motion that interpolates in 3D between the two congruent shapes, and (4) at each frame, a perspective projection of a user-selected portion of the original image. The animation may be modified interactively by adjusting the final positions of the lines or the focal length. We suggest applications to the animation of hand-drawn scenes, to the morph between two photographs, and to the intuitive design of camera motions for indoor and street scenes.
CineFilter: Unsupervised Filtering for Real Time Autonomous Camera Systems
(The Eurographics Association, 2020) Achary, Sudheer; Moorthy, K. L. Bhanu; Javed, Ashar; Shravan, Nikitha; Gandhi, Vineet; Namboodiri, Anoop M.; Christie, Marc and Wu, Hui-Yin and Li, Tsai-Yen and Gandhi, Vineet
Autonomous camera systems are often subjected to an optimization operation to smoothen and stabilize the rough trajectory estimates. Most common filtering techniques do reduce the irregularities in data; however, they fail to mimic the behavior of a human cameraman. Global filtering methods modeling human camera operators have been successful; however, they are limited to offline settings. In this paper, we propose two online filtering methods called Cinefilters, which produce smooth camera trajectories that are motivated by cinematographic principles. The first filter (CineConvex) uses a sliding windowbased convex optimization formulation, and the second (CineCNN) is a CNN based encoder-decoder model. We evaluate the proposed filters in two different settings, namely a basketball dataset and a stage performance dataset. Our models outperform previous methods and baselines on quantitative metrics. The CineConvex and CineCNN filters operate at about 250fps and 1000fps, respectively, with a minor latency (half a second), making them apt for a variety of real-time applications.
Comparing Film-editing
(The Eurographics Association, 2015) Galvane, Quentin; Ronfard, Rémi; Christie, Marc; W. Bares and M. Christie and R. Ronfard
Through a precise 3D animated reconstruction of a key scene in the movie ""Back to the Future"" directed by Robert Zemekis, we are able to make a detailed comparison of two very different versions of editing. The first version closely follows film editor Arthur Schmidt original sequence of shots cut in the movie. The second version is automatically generated using our recent algorithm [GRLC15] using the same choice of cameras. A shot-by-shot and cut-by-cut comparison demonstrates that our algorithm provides a remarkably pleasant and valid solution, even in such a rich narrative context, which differs significantly from the original version more than 60% of the time. Our explanation is that our version avoids stylistic effects while the original version favors such effects and uses them effectively. As a result, we suggest that our algorithm can be thought of as a baseline (""film-editing zero degree"") for future work on film-editing style.
A Computational Framework for Vertical Video Editing
(The Eurographics Association, 2015) Gandhi, Vineet; Ronfard, Rémi; W. Bares and M. Christie and R. Ronfard
Vertical video editing is the process of digitally editing the image within the frame as opposed to horizontal video editing, which arranges the shots along a timeline. Vertical editing can be a time-consuming and error-prone process when using manual key-framing and simple interpolation. In this paper, we present a general framework for automatically computing a variety of cinematically plausible shots from a single input video suitable to the special case of live performances. Drawing on working practices in traditional cinematography, the system acts as a virtual camera assistant to the film editor, who can call novel shots in the edit room with a combination of high-level instructions and manually selected keyframes.
Computer Generation of Filmic Discourse from a Cognitive/Affective Perspective
(The Eurographics Association, 2015) Bateman, John; Christie, Marc; Ranon, Roberto; Ronfard, Remi; Smith, Tim; W. Bares and M. Christie and R. Ronfard
In this position paper, we argue that advances in intelligent cinematography require better models of the multimodal structure of filmic discourse, and of the inferences made by an audience while films are being watched. Such questions have been addressed by film scholars and cognitive scientists in the past, but their models have not so far had sufficient impact on the intelligent cinematography community. In the future, this community should become more interested in understanding how cinematography and editing affect the movie in the audience's mind. At the same time, such frameworks can help researchers in computer graphics use computer simulations to build experiments in film cognition and test hypotheses in film theory.
Consistent Multi- and Single-View HDR-Image Reconstruction from Single Exposures
(The Eurographics Association, 2022) Mohan, Aditya; Zhang, Jing; Cozot, Remi; Loscos, Celine; Ronfard, Rémi; Wu, Hui-Yin
Recently, there have been attempts to obtain high-dynamic range (HDR) images from single exposures and efforts to reconstruct multi-view HDR images using multiple input exposures. However, there have not been any attempts to reconstruct multi-view HDR images from multi-view Single Exposures to the best of our knowledge. We present a two-step methodology to obtain color consistent multi-view HDR reconstructions from single-exposure multi-view low-dynamic-range (LDR) Images. We define a new combination of the Mean Absolute Error and Multi-Scale Structural Similarity Index loss functions to train a network to reconstruct an HDR image from an LDR one. Once trained we use this network to multi-view input. When tested on single images, the outputs achieve competitive results with the state-of-the-art. Quantitative and qualitative metrics applied to our results and to the state-of-the-art show that our HDR expansion is better than others while maintaining similar qualitative reconstruction results. We also demonstrate that applying this network on multi-view images ensures coherence throughout the generated grid of HDR images.
Contact Visualization
(The Eurographics Association, 2016) Marvie, Jean-Eudes; Sourimant, Gael; Dufay, A.; M. Christie and Q. Galvane and A. Jhala and R. Ronfard
We present in this paper a production-oriented technique designed to visualize contact in real-time between 3D objects. The motivation of this work is to provide integrated tools in the production workflow that help artists setting-up scenes and assets without undesired floating objects or inter-penetrations. Such issues can occur easily and remain unnoticed until shading and/or lighting stages are set-up, leading to retakes of the modeling or animation stages. With our solution, artists can visualize in real-time contact between 3D objects while setting-up their assets, thus correcting earlier such misalignments. Being based on a cheap post-processing shader, our solution can be used even on low-end GPUs.
Declarative Spatial Reasoning for Intelligent Cinematography
(The Eurographics Association, 2017) Bhatt, Mehul; Schultz, Carl; Suchan, Jakob; Walega, Przemyslaw; William Bares and Vineet Gandhi and Quentin Galvane and Remi Ronfard
We present computational visuo-spatial representation and reasoning from the viewpoint of the research areas of artificial intelligence, spatial cognition and computation, and human-computer interaction. The particular focus is on demonstrating recent advances in the theory and practice of spatial reasoning, and its significance and potential as a foundational AI method for (intelligent) computational cinematography & editing systems.
Design of an Intelligent Navigation System for Participative Computer Animation
(The Eurographics Association, 2017) Liu, Iou-Shiuan; Li, Tsai-Yen; Christie, Marc; William Bares and Vineet Gandhi and Quentin Galvane and Remi Ronfard
In this paper, we will propose a novel way of interactive entertainment, called Participative Computer Animation, allowing a user to participate in a computer animated story as an observer. We consider this form of entertainment as a kind of interactive storytelling, in which the presentation and perception of the story is under the control of a user through a first-person camera. As the animation of the story unfolds, the user needs to follow and view the relevant events, a complex task which requires him to navigate in the 3D environment, and hence reduce his immersion. We therefore propose to design an intelligent navigation mechanism, in which the system can voluntarily assist the user in reaching some designated best view configurations under time constraint. We have implemented such a system and invited a few users in a pilot study to evaluate the system and provide feedback. The experimental results show that our participative computer animation system can enhance the sense of presence while the intelligent navigation mechanism can improve the quality of perceiving the animated story.
Designing an Adpative Assisting Interface for Learning Virtual Filmmaking
(The Eurographics Association, 2020) Wu, Qiu-Jie; Kuo, Chih-Hsuan; Wu, Hui-Yin; Li, Tsai-Yen; Christie, Marc and Wu, Hui-Yin and Li, Tsai-Yen and Gandhi, Vineet
In this paper, we present an adaptive assisting interface for learning virtual filmmaking. The design of the system is based on the scaffolding theory, to provide timely guidance to the user in the form of visual and audio messages that are adapted to each person's skill level and performance. The system was developed on an existing virtual filmmaking setup. We conducted a study with 24 participants, who were asked to operate the film set with or without our adaptive assisting interface. Results suggest that our system can provide users with a better learning experience and positive knowledge harvest.
Designing Computer Based Archaeological 3D-Reconstructions: How Camera Zoom Influences Attention
(The Eurographics Association, 2015) Glaser, Manuela; Lengyel, Dominik; Toulouse, Catherine; Schwan, Stephan; W. Bares and M. Christie and R. Ronfard
Previous empirical literature [Sal94; WB00] indicates that there is an attention guiding effect of zooms. In order to substantiate this conclusion, an eye-tracking study was conducted to examine the influence of camera zoom on attention processes of the recipients.
Efficient Salient Foreground Detection for Images and Video using Fiedler Vectors
(The Eurographics Association, 2015) Perazzi, Federico; Sorkine-Hornung, Olga; Sorkine-Hornung, Alexander; W. Bares and M. Christie and R. Ronfard
Automatic detection of salient image regions is a useful tool with applications in intelligent camera control, virtual cinematography, video summarization and editing, evaluation of viewer preferences, and many others. This paper presents an effective method for detecting potentially salient foreground regions. Salient regions are identified by eigenvalue analysis of a graph Laplacian that is defined over the color similarity of image superpixels, under the assumption that the majority of pixels on the image boundary show non-salient background. In contrast to previous methods based on graph-cuts or graph partitioning, our method provides continuously-valued saliency estimates with complementary properties to recently proposed color contrast-based approaches. Moreover, exploiting discriminative properties of the Fiedler vector, we devise an SVM-based classifier that allows us to determine whether an image contains any salient objects at all, a problem that has been largely neglected in previous works. We also describe how the per-frame saliency detection can be extended to improve its spatiotemporal coherence when computed on video sequences. Extensive evaluation on several datasets demonstrates and validates the state-of-the-art performance of the proposed method.
Evaluation of Deep Pose Detectors for Automatic Analysis of Film Style
(The Eurographics Association, 2022) Wu, Hui-Yin; Nguyen, Luan; Tabei, Yoldoz; Sassatelli, Lucile; Ronfard, Rémi; Wu, Hui-Yin
Identifying human characters and how they are portrayed on-screen is inherently linked to how we perceive and interpret the story and artistic value of visual media. Building computational models sensible towards story will thus require a formal representation of the character. Yet this kind of data is complex and tedious to annotate on a large scale. Human pose estimation (HPE) can facilitate this task, to identify features such as position, size, and movement that can be transformed into input to machine learning models, and enable higher artistic and storytelling interpretation. However, current HPE methods operate mainly on non-professional image content, with no comprehensive evaluation of their performance on artistic film. Our goal in this paper is thus to evaluate the performance of HPE methods on artistic film content. We first propose a formal representation of the character based on cinematography theory, then sample and annotate 2700 images from three datasets with this representation, one of which we introduce to the community. An in-depth analysis is then conducted to measure the general performance of two recent HPE methods on metrics of precision and recall for character detection , and to examine the impact of cinematographic style. From these findings, we highlight the advantages of HPE for automated film analysis, and propose future directions to improve their performance on artistic film content.
Exploring the Impact of 360º Movie Cuts in Users' Attention
(The Eurographics Association, 2020) Marañes, Carlos; Gutierrez, Diego; Serrano, Ana; Christie, Marc and Wu, Hui-Yin and Li, Tsai-Yen and Gandhi, Vineet
Virtual Reality (VR) has become more relevant since the first devices for personal use became available on the market. New content has emerged for this new medium with different purposes such as education, traning, entertainment, etc. However, the production workflow of cinematic VR content is still in an experimental phase. The main reason is that there is controversy between content creators on how to tell a story effectively. Unlike traditional filmmaking, which has been in development for more than 100 years, movie editing in VR has brought new challenges to be addressed. Now viewers have partial control of the camera and can watch every degree of the 360º that surrounds them, with the possibility of losing important aspects of the scene that are key to understand the narrative of the movie. Directors can decide how to edit the film by combining the different shots. Nevertheless, depending on the scene before and after the cut, viewers' behavior may be influenced. To address this issue, we analyze users' behavior through cuts in a professional movie, where the narrative plays an important role, and derive new insights that could potentially influence VR content creation, informing content creators about the impact of different cuts in viewers' behavior.
Film Ties: A Web-based Virtual 3D Lab for Teaching the Film Art from Script to Blocking
(The Eurographics Association, 2017) Bares, William; Requierme, Caroline; Obisesan, Elizabeth; William Bares and Vineet Gandhi and Quentin Galvane and Remi Ronfard
Film production education programs include hands-on training in script writing, planning blocking of performers and cameras on set, camera operation, editing to select the best footage at each beat of the story, and expert critiques to help students improve their work. Unfortunately, this ideal form of active, hands-on learning for film production requires access to specialized equipment, movie sets, and expert film instructors. Complementary film studies education programs teach how to read the visual language of a film to breakdown each shot to understand how and why it works. Both film production and theory education involve a social component in which students collectively screen, critique, and breakdown shots seen in the films. This short paper presents work in progress to develop a Web-based virtual 3D lab, which can be used to simulate the central learning activities found in film production and film studies educational programs. The system can also be used to crowd source annotated corpora of film, which would serve as a resource for film scholars and machine-learning algorithms.

Browse

Browsing WICED: Eurographics Workshop on Intelligent Cinematography and Editing by Title

Results Per Page

Sort Options