The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume XLIII-B2-2022
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2022, 359–366, 2022
https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-359-2022
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2022, 359–366, 2022
https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-359-2022
 
30 May 2022
30 May 2022

DEEP LEARNING FOR 3D BUILDING RECONSTRUCTION: A REVIEW

M. Buyukdemircioglu1,2,4, S. Kocaman2,3, and M. Kada4 M. Buyukdemircioglu et al.
  • 1Hacettepe University, Graduate School of Science and Engineering, Ankara, Turkey
  • 2Hacettepe University, Department of Geomatics Engineering, Ankara, Turkey
  • 3ETH Zurich, Institute of Geodesy and Photogrammetry, 8093 Zurich, Switzerland
  • 4Technische Universität Berlin, Institute of Geodesy and Geoinformation Science, Berlin, Germany

Keywords: Deep Learning, Machine Learning, 3D City Models, 3D Reconstruction, LoD

Abstract. 3D building reconstruction using Earth Observation (EO) data (aerial and satellite imagery, point clouds, etc.) is an important and active research topic in different fields, such as photogrammetry, remote sensing, computer vision and Geographic Information Systems (GIS). Nowadays 3D city models have become an essential part of 3D GIS environments and they can be used in many applications and analyses in urban areas. The conventional 3D building reconstruction methods depend heavily on the data quality and source; and manual efforts are still needed for generating the object models. Several tasks in photogrammetry and remote sensing have been revolutionized by using deep learning (DL) methods, such as image segmentation, classification, and 3D reconstruction. In this study, we provide a review on the state-of-the-art machine learning and in particular the DL methods for 3D building reconstruction for the purpose of city modelling using EO data. This is the first review with a focus on object model generation based on the DL methods and EO data. A brief overview of the recent building reconstruction studies with DL is also given. We have investigated the different DL architectures, such as convolutional neural networks (CNNs), generative adversarial networks (GANs), and the combinations of conventional approaches with DL in this paper and reported their advantages and disadvantages. An outlook on the future developments of 3D building modelling based on DL is also presented.