<link rel="stylesheet" href="styles.f3b1fba60ec7970c.css">

Publication:
Use of affective visual information for summarization of human-centric videos

Loading...
Thumbnail Image

School / College / Institute

Item type:Organizational Unit,
Item type:Organizational Unit,
Item type:Organizational Unit,

Program

Organization Authors

Co-Authors

Date

Language

Embargo Status

Journal Title

Journal ISSN

Volume Title

Alternative Title

Abstract

The increasing volume of user-generated human-centric video content and its applications, such as video retrieval and browsing, require compact representations addressed by the video summarization literature. Current supervised studies formulate video summarization as a sequence-to-sequence learning problem, and the existing solutions often neglect the surge of the human-centric view, which inherently contains affective content. In this study, we investigate the affective-information enriched supervised video summarization task for human-centric videos. First, we train a visual input-driven state-of-the-art continuous emotion recognition model (CER-NET) on the RECOLA dataset to estimate activation and valence attributes. Then, we integrate the estimated emotional attributes and their high-level embeddings from the CER-NET with the visual information to define the proposed affective video summarization (AVSUM) architectures. In addition, we investigate the use of attention to improve the AVSUM architectures and propose two new architectures based on temporal attention (TA-AVSUM-GRU) and spatial attention (SA-AVSUM-GRU). We conduct video summarization experiments on the TvSum and COGNIMUSE datasets. The proposed temporal attention-based TA-AVSUM architecture attains competitive video summarization performances with strong improvements for the human-centric videos compared to the state-of-the-art in terms of F-score, self-defined face recall, and rank correlation metrics. © 2010-2012 IEEE.

Source

Publisher

Institute of Electrical and Electronics Engineers Inc.

Citation

item.page.haspartof

Source

IEEE Transactions on Affective Computing

item.page.ispartofseries

item.page.edition

DOI

10.1109/TAFFC.2022.3222882

item.page.datauri

item.page.link

Rights

Copyrights Note

Endorsement

Review

Supplemented By

Referenced By

Related Patent

Related Goal

Google Scholar
Scholar'da Ara ↗
Altmetric
Dimensions
PlumX Metrikleri
BIP! Indicators