FROM PANORAMIC PHOTOS TO A LOW-COST PHOTOGRAMMETRIC WORKFLOW FOR CULTURAL HERITAGE 3D DOCUMENTATION

: The research aims to optimize a workflow of architecture documentation: starting from panoramic photos, tackling available instruments and technologies to propose an integrated, quick and low-cost solution of Virtual Architecture . The broader research background shows how to use spherical panoramic images for the architectural metric survey. The input data (oriented panoramic photos), the level of reliability and Image-based Modeling methods constitute an integrated and flexible 3D reconstruction approach: from the professional survey of cultural heritage to its communication in virtual museum. The proposed work results from the integration and implementation of different techniques (Multi-Image Spherical Photogrammetry, Structure from Motion, Image-based Modeling) with the aim to achieve high metric accuracy and photorealistic performance. Different documentation chances are possible within the proposed workflow: from the virtual navigation of spherical panoramas to complex solutions of simulation and virtual reconstruction. VR tools make for the integration of different technologies and the development of new solutions for virtual navigation. Image-based Modeling techniques allow 3D model reconstruction with photo realistic and high-resolution texture. High resolution of panoramic photo and algorithms of panorama orientation and photogrammetric restitution vouch high accuracy and high-resolution texture. Automated techniques and their following integration are subject of this research. Data, advisably processed and integrated, provide different levels of analysis and virtual reconstruction joining the photogrammetric accuracy to the photorealistic performance of the shaped surfaces. Lastly, a new solution of virtual navigation is tested. Inside the same environment, it proposes the chance to interact with high resolution oriented spherical panorama and 3D reconstructed model at once.


INTRODUCTION
Nowadays there are different approaches to architecture documentation and survey, more or less direct, effective and affordable in terms of economy and user professionalism.Among them, panoramic photography has strong impact and diffusion.The work here presented comes from a broader research background (Fangi, 2007(Fangi, -2012)), studying how multiple spherical images can be a low-cost and complete source of documentation and architecture survey, when used as starting point for different techniques, mostly Multi-Image Spherical Photogrammetry (MISP) and Structure from Motion (SfM).Integrating these different restitution solutions is possible to optimize the model according to the final purpose (GIS publication, museum communication, rigorous geometric survey).It is possible to augment virtual reconstructions with high photorealistic contents (Fangi et al., 2009) (Figure 1) using Image-based Modelling techniques.The chance to start with the only panoramic images and match different market solutions (interactive virtual tour, rigorous restitutions, high resolution video recording, etc.) gains weight to the here presented research.Hereafter it is given an overview of all the techniques mixed up in the proposed working pipeline: opening Panoramic Photos, MISP and SfM approaches, VR linking tools.The aim is to underline the key features driving to their choice and integration.Panoramic photography allows, by means of a good experience and stitching software, several advantages: quick and low-cost acquisition, complete shots, high resolution, field of view up to 360°, lack of distortion and chromatic aberration (Shum & Szeliski, 2001).Thanks to a simple instrumentation composed of a camera, a common computer and a distance meter it is possible to give geometric value to the panoramic images.This has driven the research with the aim to implement a stand-alone pipeline of 3D survey starting from these panoramic images and going on and over their common interactive VR navigation.

Multi-Image Spherical Photogrammetry
Multi-Image Spherical Photogrammetry (MISP), tested over the last years (Fangi, 2007(Fangi, -2012)), has already shown how spherical photos are a quick and exhaustive source of data acquisition for accurate geometric restitutions.Several are the advantages of this approach: possibility to perform a plotting with very simple measurements (like tape distances), inexpensiveness and speed, documentation completeness, drastic cut in traditional photograms and photogrammetric models (every couple of panorama includes or replaces many photogrammetric models).Thanks to the possibility not to employ expensive instrumentation as laser scanners and theodolites and the capability to perform the plotting in relative orientation, the procedure well fits for the cultural heritage survey and the documentation, especially in developing countries.All these mentioned characteristics make the Spherical Photogrammetry affordable for all and a precious tool for the survey and the documentation of cultural heritage disappearing faster than it can be documented.MISP assures a low-cost procedure characterized by:  No theodolite and total station  No signals  Low expense (less the 1000 euro)  Completeness of documentation  Great speed of execution Additional advantage is the possibility, after the panorama orientation, to use the computed orientation parameters to place in the 3D space the panoramas themselves that can be linked and navigate.Moreover, during the modeling step, the oriented panorama can be used as high-resolution texture to be projected on the 3D model itself (d'Annibale et al., 2009).

Structure from Motion
The photogrammetric Structure from Motion (Snavely et al., 2006(Snavely et al., , 2007) ) and automatic restitution (Furukawa, 2007) techniques are by now wide spread even if hard to control and trust in term of metric reliability.Strong point is their low-cost feature and the possibility to use them as useful aid for the modeling of complex surfaces (such as bas-reliefs) otherwise wasteful for the only Spherical Photogrammetry.Moreover, working with panoramic images, it is possible to improve SfM results with fewer pics characterized by geometric quality and high resolution (d'Annibale, 2011).

VR Systems
Nowadays commercial and popular software allow the interactive visualization of panoramic photos.Nevertheless own and open-ended VR tools are developed to be customizable and interfacing with other techniques and to assure an accurate control of the navigation parameters.These fit for different purposes: from virtual tour to photogrammetric data management (d'Annibale, 2010).These SfM automatic procedures are useful for the modeling of complex surfaces otherwise wasteful or impossible for the only MISP enable to well identify corresponding points.

WORKFLOW
The challenge here is to combine the MISP metric accuracy with the possibility to automate some restitution steps, widening in this way the field of the geometries feasible to be surveyed.
Relevant is also the effort to really integrate the automated reconstruction techniques into a rigorous workflow of geometric reconstruction.Lastly, solutions are implemented to interactive navigate the 3D model together with the related oriented panoramas.To give an experimental example, hereafter is described the survey experience of Coliseum in Rome (March 2012).The acquisition step is quick, low-cost and purely photographic: few field acquisition hours with a Canon EOS 60D camera and a spherical head.It consists of panoramic images that must be mutually oriented in order to cover the object occlusion areas through two different panoramas at least (Figure 5).Eleven panoramas are shot, according to the acquisition scheme in Figure 6: six partial panoramas are acquired to cover the Coliseum outside and five complete ones to acquire the inside.
The test had also the special aim of covering with a single acquisition (of five panoramas taken from the central arena) the whole Coliseum perimeter, with main ellipsoidal axes respectively of 187,5 m and 156,5 m.The proposed flowchart goes on using the equirectangular (or spherical) projections of panoramic photos as starting point for the different working steps, hereafter described.

VR Virtual Navigation
A proprietary tool (d'Annibale, 2010) allows a first interactive visualization of panoramas (Figure 7) and in the same time gives the possibility to save high-resolution rectilinear planar projections of the scene (Figure 8 and Figure 9).These shots are used in the following Structure from Motion step.The acquisition does not limit itself to the screen resolution but it can be augmented up to a tested maximum resolution of 8192x8192 px, corresponding approximately to 64 MP.Further advantage is the lack of distortion even in case of shots taken using focal lengths with wide field of view.

Multi-Image Spherical Photogrammetry (MISP) and 3D Geometric Restitution
Working with proprietary tools (Fangi, 2007(Fangi, , 2008(Fangi, , 2009(Fangi, , and 2011)), the object can be reconstructed step by step:  Photogrammetric model formation by coplanarity condition between two panoramas, applied using 4 tie points at least  Absolute orientation by means of a 3D scaling rotationtranslation  Concatenation of the model with adjacent ones  Final block bundle adjustment with a minimum of three fixed points (by collinearity)  3D object reconstruction taking place with the normal surveying methods of intersection of homologous projective lines from oriented panoramas (Figure 10).The inner orientation is skipped because the image distortion is automatically corrected by the stitching software itself, by merging the overlapping image frames in the rendering phase.The 3D model scale is simply given trough a direct measure through which is possible to fix the object reference system for the absolute orientation.The achieved accuracy is in the order of few centimeters (Table 1).

Structure from Motion (SfM) and 3D Point Restitution
The rectilinear projections extracted from the panoramas lack lens distortions and are so suitable to be elaborated by open source toolkits based on Bundler (Snavely et al., 2006(Snavely et al., , 2007) ) and PMVS2 (Furukawa, 2007) (Figure 11, Table 2).Six images are extracted by means of the above described VR tool to partially survey outside the Coliseum (resolution 5120x3840 px) and thirty-five for inside (resolution 4320x2880 px).The automatically restituted point clouds are dense and with unknown scale (Figure 12).Thanks to a similitude transformation, they can be adapted to the photogrammetric model.Experimental results are good: the automated restitution is complete also for far away objects (Figure 13, Figure 14).The panorama orientation procedure allows to link together non mutually visible panoramas on the only condition to have four shared tie points.This allows quite easily to link the exterior panoramas with the internal ones.

3D Reconstruction
After having computed the panoramas' orientation parameters, primitives can be restituted by MISP.With the help of the point cloud, it is then possible to 3D reconstruct the object by means of Image-based Modeling techniques (d'Annibale, 2009(d'Annibale, , 2011)).The aim is to build object surfaces with the photo-realistic texture coming from the same high-resolution panoramas oriented in the 3D space.Figures 15 and 16 show all the elements that are useful for the 3D photorealistic model reconstruction: in red the photogrammetric restituted geometries, the optimized mesh coming from the 3D SfM point cloud and lastly the photorealistic surfaces obtained by spherical projection of the oriented panoramas.The coexistence of more restitution systems in a single 3D modeling environment underlines the consistence of the obtained results.Figure 15

Interactive Navigation
A VR own tool and a customized graphic interface allow to interactive visualize the reconstructed 3D model and the high resolution oriented panoramas (Figure 17).This interactive exploration augments the common VR panoramas' navigation alternating it with the free surfing of the reconstructed object itself.
To have a better comprehension of the architecture, an appropriate information system can be linked and different methods of interaction can be integrated (d'Annibale, 2010).

CONCLUSIONS
The described workflow based on panoramic photos and photogrammetric techniques turns out to be complete, flexible and able to assure high resolution and quality of the photographic documentation, quick and low-cost acquisition, geometric reliability of the photogrammetric restitution, expedition and SfM automatisms.
Handling each working step, the workflow can be optimized according to the available instruments, the survey duration, the mission costs and the output characteristics.Using own tools it is possible to further customize the presented pipeline.As example it is given the study of an immersive space as complex solution to interactively visualize the reconstructed virtual scenarios (Figure 18) and provide high communicative impact.
Figure 18.VR system for virtual navigation immersive space

Figure 1 .
Figure 1.3D model (low-poly) with photorealistic texture from oriented spherical panoramas Aim of this work is to show a working pipeline that starts from data acquisition (panoramic photos) to propose an integrated, low-cost and expeditious solution of Virtual Architecture.The focus is an open-ended workflow well fitting for several survey experiences differing in: scale, object complexity, geometric linearity, distances from the object to survey, problems in field acquisition and 3D reconstruction purposes.Previous experiences tested performance, limitations and integration of the main involved restitution techniques: Multi-Image Spherical Photogrammetry (MISP) and Structure from Motion (SfM).Hereafter two different case studies are shown (Figures 2 and 3) to underline how the two mentioned techniques differ in terms of input data and restitution results (3D geometric primitives by MISP and automated point clouds by SfM).

Figure 5 .
Figure 5. Acquisition instrument and panoramic photos

Figure 11 .
Figure 11.Six high resolution shots

Figure 12 .
Figure 12.Automatic restitution of 3D points by SfM and 16 show only quick (few minutes) and first stages of the modelling step.

Figure 17 .
Figure 17.Interactive exploration of a panorama in a single 3D environment where the oriented panoramas and the 3D model are visualized at the same time

Table 1 .
Orientation step and accuracy check Figure 10.Control Network with point stations and restitution red lines plotted by Spherical Photogrammetry

Table 2 .
Planar projection information