Deep stroke-based sketched symbol reconstruction and segmentation

Abstract

Hand-drawn objects usually consist of multiple semantically meaningful parts. In this article, we propose a neural network model that segments sketched symbols into stroke-level components. Our segmentation framework has two main elements: a fixed feature extractor and a multilayer perceptron (MLP) network that identifies a component based on the feature. As the feature extractor we utilize an encoder of a stroke-rnn, which is our newly proposed generative variational auto-encoder (VAE) model that reconstructs symbols on a stroke-by-stroke basis. Experiments show that a single encoder could be reused for segmenting multiple categories of sketched symbols with negligible effects on segmentation accuracies. Our segmentation scores surpass existing methodologies on an available small state-of-the-art dataset. Moreover, extensive evaluations on our newly annotated big dataset demonstrate that our framework obtains significantly better accuracies as compared to baseline models. We release the dataset to the community.