The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume XLIII-B2-2020
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2020, 255–261, 2020
https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-255-2020
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2020, 255–261, 2020
https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-255-2020

  12 Aug 2020

12 Aug 2020

MARKER-LESS MOBILE AUGMENTED REALITY APPLICATION FOR MASSIVE 3D POINT CLOUDS AND SEMANTICS

A. Kharroubi, R. Billen, and F. Poux A. Kharroubi et al.
  • Geomatics Unit, University of Liège, Allée du six Août, 19, 4000 Liège, Belgium

Keywords: Augmented Reality, Web-AR, Marker-Less AR, 3D Point cloud, Semantics, Classification, Virtual environment

Abstract. Mobile Augmented Reality (MAR) attracts significant research and development efforts from both the industry and academia, but rarely integrate massive 3D dataset’s interactions. The emergence of dedicated AR devices and powerful Software Development Kit (ARCore for android and ARKit for iOS) improves performance on mobile devices (Smartphones and tablets). This is aided by new sensor integration and advances in computer vision that fuels the development of MAR. In this paper, we propose a direct integration of massive 3D point clouds with semantics in a web-based marker-less mobile Augmented Reality (AR) application for real-time visualization. We specifically investigate challenges linked to point cloud data structure and semantic injection. Our solution consolidates some of the overarching principles of AR, of which pose estimation, registration and 3D tracking. The developed AR system is tested on mobile phones web-browsers providing clear insights on the performance of the system. Promising results highlight a number of frame per second varying between 27 and 60 for a real-time point budget of 4.3 million points. The point cloud tested is composed of 29 million points and shows how our indexation strategy permits the integration of massive point clouds aiming at the point budget. The results also gives research directions concerning the dependence and delay related to the quality of the network connection, and the battery consumption since portable sensors are used all the time.