Optimum scaling operator selection in scalable video coding

Abstract

Scalable video coders provide different options, such as temporal, spatial and SNR scalability, where each option results in different kinds and/or levels of visual distortion at the lower scales depending on the content and bitrate. We observe that in most cases a single scalability option does not fit the whole video content well, and the scalability operator should be varied for different temporal segments depending on the content of the segment. In this work, assuming the video is temporally segmented by some content analysis scheme, we propose a method to choose the visually best scaling option that results in minimum visual distortion among temporal, spatial and SNR scalability operators for each temporal segment of soccer videos. We employ four component metrics to quantify artifacts caused by bitrate reduction, spatial size reduction and temporal subsampling, which are a flatness measure, a blockiness measure, a blurriness measure, and a temporal distortion (jerkiness) measure. We then define the best scaling operator for each video segment as the one with the minimum distortion score which is given by a linear combination of these four component measures. Coefficients of this linear combination are tuned to content type using a training procedure. Two subjective tests have been performed to validate the proposed distortion measures and procedure for optimal selection of scalability operators for soccer videos.