PROJECT PHOTOFLY : NEW 3D MODELING ONLINE WEB SERVICE (CASE STUDIES AND ASSESSMENTS)

During summer 2010, Autodesk has released a still ongoing project called Project Photofly, freely downloadable from AutodeskLab web site until August 1 2011. Project Photofly based on computer-vision and photogrammetric principles, exploiting the power of cloud computing, is a web service able to convert collections of photographs into 3D models. Aim of our research was to evaluate the Project Photofly, through different case studies, for 3D modeling of cultural heritage monuments and objects, mostly to identify for which goals and objects it is suitable. The automatic approach will be mainly analyzed.


INTRODUCTION
Sometimes it is not easy to digitize cultural heritage artifacts, due to permits granted by public authority, environmental features, limited time available on site, and so on and so forth.In these cases the speed of acquisition is a key factor during field survey and photography ensures it.The final goal of the project must also be taken into account: visualization purposes for large audiences; restoration; recording and documentation.Indeed expensive and time consuming photogrammetric approaches or laser scanning survey are not worth if the final goal of the project is just visualization, and high accuracy of the 3D model is not required.
Photofly technology was acquired on May 2008 from Realviz.After few years of research and development conducted by AutodeskLab, Photofly was released on July 22 2010.Realviz, one of the few available commercial technologies, developed the image processing software products, which enabled the production of 3D digital content of a set of images.
Structure from motion (SfM) is the main concept which stands behind a fully automated 3D modelling approach from set of markless images.Recently some tools are available.In 2006 M. Vergauwen and L. Van Gool released a Web-based 3D reconstruction service (Vergauwen and Van Gool, 2006).The automatic process, running on a remote server, computes the camera calibration together with the depth maps of the images uploaded.The result is then downloaded on the user's PC.Other two projects which exploit the SfM concepts are Bundler, developed at the University of Washington, USA (Snavely et al. 2008) and Samantha developed at the University of Verona, Italy (Farenzena et al. 2009).Both are able to deal with huge number of images.Bundler is SfM system for unordered image collections able to create sparse point clouds.An earlier version of it was used for the Photo Tourism project (Snavely et al. 2008).To obtain denser point clouds Bundler can be used in conjunction with PMVS2 (Furukawa and Ponce 2007).Samantha appears to be even faster than Bundler.It shows a lower computational complexity and introduces a local bundle adjustment procedure.A new methodology for the automatic identification of an images correspondence, for a wide range of image datasets, is represented by ATiPE.The acronym stands for Automatic Tie Point Extraction (Barazzetti et al. 2010).It distinguishes itself from other software packages because of its flexibility, capability to process different kinds of images and completeness of the obtained point cloud.

Project Photofly. How it works.
Project Photofly is a web service technology .The user has to install the "Photo Scene Editor" to upload images, and possibly to modify manually the photo-scene returned by the system.Although Pictures can be shot with various camera and various focal length, it's recommended to use the same camera, even a commercial camera, and the same zoom ratio for the entire scene dataset.Better results are gained using a wide angle lens (20 to28 mm or more).Fish eye lenses are not supported by the Photofly engine, whereas ultra wide lenses may create a distortion in the images, which could not be compensated during the reconstruction.During the photographs acquisition it is important to avoid objects occluding the target, and objects in movements in front of it, as well as flat and untextured surfaces which wouldn't help to create the depth information.Once the photographs (jpeg or tif file formats), taken with standard digital cameras and following specific guidelines, are uploaded to the server, the system computes the photo-scene and sends a sparse point cloud back to the user.He can decide to wait online until the process is completed by the server, and the file is sent back, or leave a valid email address, where he will be contacted later on, to download the 3D model (rzi file format).Basically the server stitches the camera positions, and as many photographs as it can together, by itself using a series of computer vision algorithms called the "Camera Factory".Additional features can be applied locally after the download, and the final model can be eventually exported.Nowadays the Photo Scene Editor is version 1.2 and the point cloud can be exported in four file formats: dwg, las, fbx, rzi.If the web service is unable to determine where some pictures belong, the user can manually identify at least four common points for each picture, in order to have the unused photo added to the global scene.It is also possible to draw lines directly onto the point cloud and export just them to a CAD software package for further changes and improvements.
The system itself is able to use metadata stored in the photographs to get the camera features and improve the accuracy of the model.If the exif file is available, it is automatically read and the images are grouped by same focal length.However the application does never ask for those instructions directly.

CASE STUDIES
Pictures datasets of different cultural heritage artifacts, environments (squares) and buildings have been acquired, according with different dimensional scales and different cameras.
Each dataset was uploaded to the Project Photofly server and the results obtained by the automated process were analyzed.

Navona Square, Rome
Navona Square is placed in the historic city centre of Rome.Its shape is an ellipse.The major axis is 250 meters and the minor axis is 50 meters.
The photographic dataset of Navona Square consists of 92 pictures.
The automatic reconstruction was unsuccessful.The engine stitched together 83 pictures on a total amount of 92.However on 83 pictures, only 44 were correctly placed whereas the other 48 were mixed with each other.It follows that the final points

ASSESSMENTS AND EVALUATION
The main object of the paper was to evaluate the Project Photofly in order to understand goals, and for which objects it is suitable, and to highlight its strengths and weaknesses.Some case studies were tested for these purposes.Different sites and objects were digitized according with different scales:

Open Spaces (City Squares): results
All the 3D reconstructions of city squares were unsuccessful.Photofly seems to have problems when dealing with such open spaces.Manual intervention was always necessary to achieve a correct result.
In the Navona Square dataset the automatic process failed probably due to some obstructions, represented by the "Fountain of the four rivers" by Bernini, which at that time was surrounded by flat white panels.However, after marking points on building behind them, the server was able to model the entire scene correctly.Also for the Campidoglio Square dataset the automatic process failed.In this case the wrong reconstruction was possibly caused by the equality of the two buildings on the sides of the square facing each other.One building was indeed placed wrongly behind its twin and with a bigger scale.
The last case study for this typology was the Maggiore Square in Bologna.Although the square was free of obstruction and there were no similar buildings the 3D reconstruction was incorrect.

Building Facades: results
One test based on facades (Palazzo d'Accursio, Bologna) was successful.Photofly server correctly reconstructed the model.It seems that it has no problems dealing with surface with no specular sides when the camera moves in just one direction.However for Palazzo dei Diamanti, where the façade has mirror sides divided by the main entrance, the resulting 3D model was wrong.The two sides were superimposed.For this building also a distortion of the point cloud was noticed.

Isolated Buildings: results
Isolated Buildings, with no obstructions occluding the camera field of view, and without other structures leaning among them, were chosen.Both the case studies showed that Photofly works correctly with isolated buildings (simple and complex shapes).However for the reconstruction of the Estense Castle it wasn't possible to model the entire site since the engine wasn't able to process all the pictures taken on field.The upload was tried different times, even subdividing the dataset in smaller groups, always with the lack of results.
Only half of the photographs database was processed.Despite this, the model obtained was correct.
The camera path around both of them was circular (360 degrees) and the photographs were taken from different distances due to specific environmental conditions.

Architectonics Elements and Sculptural Objects: results
Simple objects, with no obstructions and easy access, were chosen for this test.
All the object digitized were modeled precisely exploiting the Photofly automatic pipeline.
For two of them (portal and bust), among four, the camera path was semicircular, covering 180 degrees in front of them.For the hippogriph the camera moved 360 degrees around it, and for the forth one (rib vault) the camera path was moving along its four edges.All pictures were stitched together and none of them was missed or added manually.

CONCLUSIONS
Based on this study and due to lack of other documentation, besides the AutodeskLab forum, the following strengths and weakness can be highlighted.

Figure 2 .
Figure 2. Navona Square, Rome, Automatic Reconstruction Project Photofly is able to create quickly models (point cloud) of objects, architectonic elements, surfaces and isolated buildings, where pictures are taken without any overwhelming obstruction or elements like moving objects, which can represent noise in the camera field of view.It shows problems to reconstruct complex and articulated spaces, as highlighted in the square typology.It experiences problems when dealing with large dataset and when the scene includes mirror elements.In these environments other techniques, like Laser Scanning, are preferable.At this point Project Photofly is still a technology preview.It is not a mature technology.At this stage, failures are as interesting as successes.