Data augmentation and U-Net CNN for accurate nuclei segmentation on Pap smear images

Abstract

The nuclei and cytoplasm can be detected through Pap smear images. To help women avoid cervical cancer, early detection of nuclei abnormalities can be known through image segmentation. U-Net CNN is known as an architecture commonly used for segmentation. U-Net CNN needs a large amount of data for training. The amount of pap smear images is limited. A small amount of data can cause overfitting and reduce the performance of U-Net. This study combines augmentation and segmentation, the augmentation in this study combines geometric transformation with flipping and rotation, and pixel-wise transform with Gamma Correction. The augmentation aims to generate new images that are more numerous and varied. The result of the proposed method is a pap smear image which only consists of two parts, the background and nuclei as the foreground. The performance evaluation of the combination of the augmentation method and U-Net CNN is accuracy, sensitivity, specificity, and F1 score. The application of augmentation using Flipping, rotation, and Gamma Correction can increase 10 times the amount of Pap Smear image data. The average results of accuracy, sensitivity, specificity, and F1 score of U-Net on augmented data are 93.75%. These results show that the combination of augmentation and U-Net CNN is excellent and robust at detecting nuclei in pap smear images. The proposed method can be developed to segment other cells such as cytoplasm and can be as a component for building an automatic cervical cancer detection. © 2024, Jurusan Teknik Elektromedik, Politeknik Kesehatan Kemenkes Surabaya, Indonesia. All rights reserved.