Online nonlinear classification for high-dimensional data

Abstract

We study online binary classification problem under the empirical zero-one loss function. We introduce a novel randomized classification algorithm based on highly dynamic hierarchical models that partition the feature space. Our approach jointly and sequentially learns the partitioning of the feature space, the optimal classifier among all doubly exponential number of classifiers defined by the tree, and the individual region classifiers in order to directly minimize the cumulative loss. Although we adapt the entire hierarchical model to minimize a global loss function, the computational complexity of the introduced algorithm scales linearly with the dimensionality of the feature space and the depth of the tree. Furthermore, our algorithm can be applied to any streaming data without requiring a training phase or prior information, hence processes data on-the-fly and then discards it, which makes the introduced algorithm significantly appealing for applications involving "big data". We evaluate the performance of the introduced algorithm over different real data sets.