Adaptive mixture methods based on Bregman divergences

Abstract

We investigate affinely constrained mixture methods adaptively combining outputs of m constituent filters running in parallel to model a desired signal. We use Bregman divergences and obtain multiplicative updates to train these linear combination weights under the affine constraints. We use the unnormalized relative entropy and the relative entropy that produce the exponentiated gradient update with unnormalized weights (EGU) and the exponentiated gradient update with positive and negative weights (EG), respectively. We carry out the mean and the mean-square transient analysis of the affinely constrained mixtures of m filters using the EGU or EG algorithms. We compare performances of different algorithms through our simulations and illustrate the accuracy of our results.