Joint Attention for Automated Video Editing

dc.contributor.authorWu, Hui-Yinen_US
dc.contributor.authorSantarra, Trevoren_US
dc.contributor.authorLeece, Michaelen_US
dc.contributor.authorVargas, Rolandoen_US
dc.contributor.authorJhala, Arnaven_US
dc.contributor.editorChristie, Marc and Wu, Hui-Yin and Li, Tsai-Yen and Gandhi, Vineeten_US
dc.date.accessioned2020-05-24T13:14:09Z
dc.date.available2020-05-24T13:14:09Z
dc.date.issued2020
dc.description.abstractJoint attention refers to the shared focal points of attention for occupants in a space. In this work, we introduce a computational definition of joint attention for the automated editing of meetings in multi-camera environments from the AMI corpus. Using extracted head pose and individual headset amplitude as features, we developed three editing methods: (1) a naive audio-based method that selects the camera using only the headset input, (2) a rule-based edit that selects cameras at a fixed pacing using pose data, and (3) an editing algorithm using LSTM (Long-short term memory) learned joint-attention from both pose and audio data, trained on expert edits. The methods are evaluated qualitatively against the human edit, and quantitatively in a user study with 22 participants. Results indicate that LSTM-trained joint attention produces edits that are comparable to the expert edit, offering a wider range of camera views than audio, while being more generalizable as compared to rule-based methods.en_US
dc.description.sectionheadersAfternoon Session
dc.description.seriesinformationWorkshop on Intelligent Cinematography and Editing
dc.identifier.doi10.2312/wiced.20201131
dc.identifier.isbn978-3-03868-127-4
dc.identifier.issn2411-9733
dc.identifier.pages37-37
dc.identifier.urihttps://doi.org/10.2312/wiced.20201131
dc.identifier.urihttps://diglib.eg.org:443/handle/10.2312/wiced20201131
dc.publisherThe Eurographics Associationen_US
dc.subjectsmart conferencing
dc.subjectautomated video editing
dc.subjectjoint attention
dc.subjectLSTM
dc.titleJoint Attention for Automated Video Editingen_US
Files
Collections