Decoding cognitive states using the bag of words model on fMRI time series

Abstract

Bag-of-words (BoW) modeling has yielded successful results in document and image classification tasks. In this paper, we explore the use of BoW for cognitive state classification. We estimate a set of common patterns embedded in the fMRI time series recorded in three dimensional voxel coordinates by clustering the BOLD responses. We use these common patterns, called the code-words, to encode activities of both individual voxels and group of voxels, and obtain a BoW representation on which we train linear classifiers. Our experimental results show that the BoW encoding, when applied to individual voxels, significantly improves the classification accuracy (an average 7.2% increase over 13 different datasets) compared to a classical multi voxel pattern analysis method. This preliminary result gives us a clue to generate a code-book for fMRI data which may be used to represent a variety of cognitive states to study the human brain.