Publication:
A gated fusion network for dynamic saliency prediction

Thumbnail Image

Departments

School / College / Institute

Program

KU-Authors

KU Authors

Co-Authors

Kocak, Aysun
Erdem, Erkut

Publication Date

Language

Embargo Status

NO

Journal Title

Journal ISSN

Volume Title

Alternative Title

Abstract

Predicting saliency in videos is a challenging problem due to complex modeling of interactions between spatial and temporal information, especially when ever-changing, dynamic nature of videos is considered. Recently, researchers have proposed large-scale data sets and models that take advantage of deep learning as a way to understand what is important for video saliency. These approaches, however, learn to combine spatial and temporal features in a static manner and do not adapt themselves much to the changes in the video content. In this article, we introduce the gated fusion network for dynamic saliency (GFSalNet), the first deep saliency model capable of making predictions in a dynamic way via the gated fusion mechanism. Moreover, our model also exploits spatial and channelwise attention within a multiscale architecture that further allows for highly accurate predictions. We evaluate the proposed approach on a number of data sets, and our experimental analysis demonstrates that it outperforms or is highly competitive with the state of the art. Importantly, we show that it has a good generalization ability, and moreover, exploits temporal information more effectively via its adaptive fusion scheme.

Source

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Subject

Computer science, Robotics, Neurosciences and neurology

Citation

Has Part

Source

IEEE Transactions on Cognitive and Developmental Systems

Book Series Title

Edition

DOI

10.1109/TCDS.2021.3094974

item.page.datauri

Link

Rights

Copyrights Note

Endorsement

Review

Supplemented By

Referenced By

1

Views

4

Downloads

View PlumX Details