The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume XLIV-2/W1-2021
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIV-2/W1-2021, 33–40, 2021
https://doi.org/10.5194/isprs-archives-XLIV-2-W1-2021-33-2021
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIV-2/W1-2021, 33–40, 2021
https://doi.org/10.5194/isprs-archives-XLIV-2-W1-2021-33-2021

  15 Apr 2021

15 Apr 2021

BOOSTING SEGMENTATION ACCURACY OF THE DEEP LEARNING MODELS BASED ON THE SYNTHETIC DATA GENERATION

V. V. Danilov1,5, O. M. Gerget1, D. Y. Kolpashchikov1, N. V. Laptev1, R. A. Manakov2, L. A. Hérnandez-Gómez3, F. Alvarez4, and M. J. Ledesma-Carbayo5 V. V. Danilov et al.
  • 1Research Laboratory for Processing and Analysis of Big Data, Tomsk Polytechnic University, Tomsk, Russia
  • 2Department of Software Engineering, Research Laboratory ”Gamma Technologies”, Almaty, Kazakhstan
  • 3Department of Signals, Systems and Radiocommunications, Technical University of Madrid, Madrid, Spain
  • 4Group of Application of Visual Telecommunications, Technical University of Madrid, Madrid, Spain
  • 5Biomedical Image Technologies Group, Technical University of Madrid, Madrid, Spain

Keywords: Data Synthesis, Echocardiography, Catheter Segmentation, Forward Kinematics, Spline Coordinate System

Abstract. In the era of data-driven machine learning algorithms, data represents a new oil. The application of machine learning algorithms shows they need large heterogeneous datasets that crucially are correctly labeled. However, data collection and its labeling are time-consuming and labor-intensive processes. A particular task we solve using machine learning is related to the segmentation of medical devices in echocardiographic images during minimally invasive surgery. However, the lack of data motivated us to develop an algorithm generating synthetic samples based on real datasets. The concept of this algorithm is to place a medical device (catheter) in an empty cavity of an anatomical structure, for example, in a heart chamber, and then transform it. To create random transformations of the catheter, the algorithm uses a coordinate system that uniquely identifies each point regardless of the bend and the shape of the object. It is proposed to take a cylindrical coordinate system as a basis, modifying it by replacing the Z-axis with a spline along which the h-coordinate is measured. Having used the proposed algorithm, we generated new images with the catheter inserted into different heart cavities while varying its location and shape. Afterward, we compared the results of deep neural networks trained on the datasets comprised of real and synthetic data. The network trained on both real and synthetic datasets performed more accurate segmentation than the model trained only on real data. For instance, modified U-net trained on combined datasets performed segmentation with the Dice similarity coefficient of 92.6±2.2%, while the same model trained only on real samples achieved the level of 86.5±3.6%. Using a synthetic dataset allowed decreasing the accuracy spread and improving the generalization of the model. It is worth noting that the proposed algorithm allows reducing subjectivity, minimizing the labeling routine, increasing the number of samples, and improving the heterogeneity.