Directed fine tuning using feature clustering for instance segmentation of toxoplasmosis fundus images

Abstract

Medical image segmentation is a challenging problem for computer vision approaches where deep learning networks have achieved impressive successes in recent years. In this paper, we propose a directed, fine tuning approach for instance segmentation networks by using feature clustering of predictions along with labeled training instances to improve network performance. The approach directs and limits analyses of predicted instances by experts to similar training instances only and reduces manual overheads by managing the number of instances that need to be examined. Sub-optimal network predictions are handled either by retraining the networks on data augmented with the relevant training instances, correcting training labels, and/or by readjusting network inference parameters. We first develop a state-of-the-art Mask R-CNN based network for instance segmentation of fundus images with retinal lesions and scars caused by Ocular Toxoplasmosis. Then, we show how the proposed approach can be applied to fine tune this network in a directed manner using feature clustering using a pre-trained CNN network. We demonstrate the robustness of our proposed approach with the evaluation results - mask average IoU increased by 7% and mAP under 0.5 IoU threshold increased by 20%. Our experiments also show that fine tuning by analyzing 66% of the predicted instances achieves the same improvement as that obtained by all of the predicted instances, a significant reduction of the manual overheads for fine tuning