The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume XLIII-B2-2020
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2020, 435–441, 2020
https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-435-2020
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2020, 435–441, 2020
https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-435-2020

  12 Aug 2020

12 Aug 2020

WIRE STRUCTURE IMAGE-BASED 3D RECONSTRUCTION AIDED BY DEEP LEARNING

V. V. Kniaz1,2, S. Y. Zheltov1, F. Remondino3, V. A. Knyaz1,2, A. Bordodymov1, and A. Gruen4 V. V. Kniaz et al.
  • 1State Res. Institute of Aviation Systems (GosNIIAS), 125319, 7, Victorenko str., Moscow, Russia
  • 2Moscow Institute of Physics and Technology (MIPT), Dolgoprudny, Russia
  • 3Bruno Kessler Foundation (FBK), Trento, Italy
  • 4ETH Zurich, Switzerland

Keywords: structure from motion, wire structures 3D reconstruction, segmentation, deep learning, Shukhov Radio tower

Abstract. Objects and structures realized by connecting and bending wires are common in modern architecture, furniture design, metal sculpting, etc. The 3D reconstruction of such objects with traditional range- or image-based methods is very difficult and poses challenges due to their unique characteristics such as repeated structures, slim elements, holes, lack of features, self-occlusions, etc. Complete 3D models of such complex structures are normally reconstructed with lots of manual intervention as automated processes fail in providing detailed and accurate 3D reconstruction results.

This paper presents the image-based 3D reconstruction of the Shukhov hyperboloid tower in Moscow, a wire structure built in 1922, composed of a series of hyperboloid sections stacked one to another to approximate an overall conical shape. A deep learning approach for image segmentation was developed in order to robustly detect wire structures in images and provide the basis for accurate corresponding problem solutions. The developed WireNet convolution neural network (CNN) model has been used to aid the multi-view stereo (MVS) process and to improve robustness and accuracy of the image-based 3D reconstruction approach, otherwise not feasible without masking the images automatically.