ARTMV: a cross-modal art music video dataset for proprioceptive valence perception

Abstract

We present a novel approach for affective multimedia content analysis to study how the human keypoints contribute to the perceived emotion of art music. Traditional music information retrieval methodologies have extensively used the cross-modal bias of audio and visual modalities to assess affective states. In the case of art music videos, the visual modality is limited by orchestra footage or static images, lacking the dynamic visual elements commonly found in videos of other music genres. In this paper, we introduce ARTMV, an art music video dataset consisting of perceived static categorical valence labels, music tracks and related dance videos. To overcome the restrictive visual content, our proposed network competitively replaces the visual modality of the videos with the proprioception of the performers from the dance performances of the corresponding art music.