Self-validated U-GAN for Target Class Transformation in Segmentation Tasks

Abstract

The paper addresses the problem of data scarcity for training automated intelligent systems in various specific fields such as medicine, satellite image analysis, agriculture, ecology, and language. It describes modern methods for solving this problem, including augmentation, generative adversarial networks, diffusion models, and inpainting. The focus is on the task of segmentation, where it is necessary to create masks for new objects in addition to the image. The subjective and manual process of selecting the best epoch during model training is also noted, and alternatives that can help solve this problem, such as inception score and frechet inception distance, are outlined.

An improved model of partial transformation of the target class of segmentation is proposed, which includes new self-validation components, such as an additional loss function that controls the similarity of the output image to the input one, a pretrained segmentation model, and a metric for assessing the quality of the generated masks with segmentation masks of generated images. These improvements allow the system to more effectively transform the background or zero class into the target one, create more accurate segmentation masks, and automatically select the best epochs during training.

Experiments on a dataset of panoramic tooth images showed that the use of this technology allowed increasing the accuracy of filling segmentation by 0.9%, raising the Jaccard coefficient value from 90.5% to 91.4%. The generative adversarial network model was trained for 150 epochs with automatic selection of the best epoch, which was the 135th epoch, and the quality of the generated images of this epoch was confirmed by expert evaluation. On satellite images of ships, the use of the model showed an improvement in segmentation accuracy from 63.4% to 65.2%. Despite the complexity of the data, the model was able to adequately transform the input data of the empty sea into ship objects. The best results were achieved at the 82nd epoch, which also coincided with the expert's choice of the best epoch, demonstrating the importance of automatic selection of the optimal epoch during training to eliminate additional subjective factors from this process and speed up model preparation.

These results confirm the effectiveness of the proposed approach, showing metrics improvements and better automation of the basic approach. The proposed methods and approaches have the potential for wide application in various fields, contributing to the development of new intelligent systems and increasing their accuracy.