Optimal and efficient distributed online learning for big data

Abstract

We propose optimal and efficient distributed online learning strategies for Big Data applications. Here, we consider the optimal state estimation over distributed network of autonomous data sources. The autonomous data sources can generate and process data locally irrespective of any centralized control unit. We seek to enhance the learning rate through the distributed control of those autonomous data sources. We emphasize that although this problem attracted significant attention and extensively studied in different fields including services computing and machine learning disciplines, all the well-known strategies achieve suboptimal online learning performance in the mean square error sense. To this end, we introduce the oracle algorithm as the optimal distributed online learning strategy. We also propose the optimal and efficient distributed online learning algorithm that reduces the communication load tremendously, i.e., requires the undirected disclosure of only a single scalar. Finally, we demonstrate the significant performance gains due to the proposed strategies with respect to the state-of-the-art approaches.