Information-theoretic framework for multimodal signal processing

Abstract

The signal processing community has shown to be increasingly reliant upon information theoretic concepts to develop algorithms for a wide variety of important problems ranging e. g. from audio-visual signal processing to medical imaging. When comparing the proposed algorithms, two facts are particularly surprising. First, the range of practical problems which, are solved with the fundamentally very compact mathematical concepts of information theory seem to be very broad and unrelated. The second striking fact is that the mathematical expressions governing the final algorithms seem not to be much related, even though the employed fundamental concepts were identical. The main aim of this chapter consists of developing an information theoretic framework for multimodal signal processing, closely related to information theoretic feature extraction/selection. This important relationship will indicate how we can unify to a large extent multimodal medical image processing, e. g. multi-channel segmentation and image registration, and extend information theoretic registration to other features than image intensities. The framework is not at all restricted to medical images though and we will illustrate this by applying it to multimedia sequences as well.