The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume XLIII-B2-2020
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2020, 857–862, 2020
https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-857-2020
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2020, 857–862, 2020
https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-857-2020

  12 Aug 2020

12 Aug 2020

MACHINE LEARNING FOR APPROXIMATING UNKNOWN FACE

V. A. Knyaz1,2, V. V. Kniaz1,2, M. M. Novikov3, and R. M. Galeev4 V. A. Knyaz et al.
  • 1State Research Institute of Aviation System (GosNIIAS), 125319 Moscow, Russia
  • 2Moscow Institute of Physics and Technology (MIPT), Dolgoprudny, Russia
  • 3Research Center Crystallography and Photonics RAS, Shatura, Russia
  • 4Institute of Ethnology and Anthropology RAS, Moscow, Russia

Keywords: photogrammetry, 3D reconstruction, facial approximation, machine learning, generative adversarial networks, anthropology

Abstract. The problem of facial appearance reconstruction (or facial approximation) basing on a skull is very important as for anthropology and archaeology as for forensics. Recent progress in optical 3D measurements allowed to substitute manual facial reconstruction techniques with computer-aided ones based on digital skull 3D models. Growing amount of data and developing methods for data processing provide a background for creating fully automated technique of face approximation.

The performed study addressed to a problem of facial approximation based on skull digital 3D model with deep learning techniques. The skull 3D models used for appearance reconstruction are generated by the original photogrammetric system in automated mode. These 3D models are then used as input for the algorithm for face appearance reconstruction. The paper presents a deep learning approach for facial approximation basing on a skull. It exploits the generative adversarial learning for transition data from one modality (skull) to another modality (face) using digital skull 3D models and face 3D models. A special dataset containing skull 3D models and face 3D models has been collected and adapted for convolutional neural network training and testing. Evaluation results on testing part of the dataset demonstrates high potential of the developed approach in facial approximation.