kCV-B: BOOTSTRAP WITH CROSS-VALIDATION FOR DEEP LEARNING MODEL DEVELOPMENT, ASSESSMENT AND SELECTION
- 1Geodesy and Geospatial Engineering, Faculty of Science, Technology and Medicine, University of Luxembourg, 6, rue Richard Coudenhove-Kalergi, L-1359, Luxembourg
- 2Center for Urban Science and Progress; Department of Civil and Urban Engineering, Tandon School of Engineering, New York University, USA
- 33D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK), Trento, Italy
- 4Department of Computer Engineering, Karabuk University, Karabuk, Türkiye
- 5Geography and Environmental Management, University of Waterloo, Waterloo ON N2L 3G1, Canada
Keywords: Classification, Cross-Validation, Neural Network, PointNet, Semantic Segmentation, Supervised Machine Learning
Abstract. This study investigates the inability of two popular data splitting techniques: train/test split and k-fold cross-validation that are to create training and validation data sets, and to achieve sufficient generality for supervised deep learning (DL) methods. This failure is mainly caused by their limited ability of new data creation. In response, the bootstrap is a computer based statistical resampling method that has been used efficiently for estimating the distribution of a sample estimator and to assess a model without having knowledge about the population. This paper couples cross-validation and bootstrap to have their respective advantages in view of data generation strategy and to achieve better generalization of a DL model. This paper contributes by: (i) developing an algorithm for better selection of training and validation data sets, (ii) exploring the potential of bootstrap for drawing statistical inference on the necessary performance metrics (e.g., mean square error), and (iii) introducing a method that can assess and improve the efficiency of a DL model. The proposed method is applied for semantic segmentation and is demonstrated via a DL based classification algorithm, PointNet, through aerial laser scanning point cloud data.