FROM METRIC IMAGE ARCHIVES TO POINT CLOUD RECONSTRUCTION : CASE STUDY OF THE GREAT MOSQUE OF ALEPPO IN SYRIA

The paper presents photogrammetric archives from Aleppo (Syria), collected between 1999 and 2002 by the Committee for maintenance and restoration of the Great Mosque in partnership with the Engineering Unit of the University of Aleppo. During that period, terrestrial photogrammetric data and geodetic surveys of the Great Omayyad mosque were recorded for documentation purposes and geotechnical studies. During the recent war in Syria, the Mosque has unfortunately been seriously damaged and its minaret has been completely destroyed. The paper presents a summary of the documentation available from the past projects as well as solutions of 3D reconstruction based on the processing of the photogrammetric archives with the latest 3D image-based techniques.


INTRODUCTION
The paper presents photogrammetric archives recorded in 1999 and 2000 by the Czech company Geodézie CS for the Committee for maintenance and restoration of the Great Mosque of Aleppo, in partnership with the Engineering Unit of the University of Aleppo (Grussenmeyer, 2016).This company carried out a series of topographical and photogrammetric work for the documentation of the Great Mosque of Aleppo and for geotechnical studies.The aim was to provide topographical maps and elevations including the planimetry and altimetry details of the four outer walls, the outer west and north walls, the two east and south entrances, and the dome of the Great Mosque.3D vector models of each structure were delivered.Complementary metric photos have been recorded in 2001 by master students from INSA Strasbourg for the monitoring of the structure of the minaret.
Those images are today valuable archives as the Mosque has unfortunately been seriously damaged during the recent conflict.The paper will present a summary of the documentation available from the past projects as well as solutions of 3D reconstruction based on the processing of point clouds from the photogrammetric archives with the latest 3D image-based techniques.

HISTORY OF THE GREAT MOSQUE OF ALEPPO
The Great Mosque or Umayyad Mosque of Aleppo, was built on the site of a former Roman temple and Byzantine cathedral built by St. Helen (Mother of Constantine the Great).The mosque was founded by the Umayyad Caliph al Walid in 715 and completed by his successor Caliph Suleiman.The Great Mosque of Aleppo is the largest and one of the oldest mosques in the city of Aleppo (Syria).It is located in al-Jalloum district of the Ancient City of Aleppo, a World Heritage Site, near the entrance to al-Madina Souq.The Mosque was built at the beginning of the 8th Century but the recent buildings date from the 11th and 14th centuries.Heavy fighting during the Syrian war have ruined the holy site.In fact, the Mosque has suffered extensive damage during months of war, with antique furnishings and intricately sculpted colonnades affected.The famous minaret, where the Muslim call to prayer sounded for 900 years, has been toppled and shattered (Figure 1).The minaret has been destroyed in 2013.

EXISTING DOCUMENTATION
In 1999, the Head of the Committee for maintenance and restoration of the Great Omayyad Mosque in Aleppo published the technical terms for the project of documentation of the Mosque by photogrammetric measurements.The object of the project was to document the Mosque by photogrammetric techniques in view of knowing the situation of the Mosque and the architectural, ornamental and building details, as well as structural and general aspects of deterioration including cracks, distortions, etc.

Technical terms of the documentation
The required works were divided in several parts: 1. Topographic plans, including planimetric and elevation details (contour lines), with basic scales of 1/10 to 1/50, interval of contour lines of (2-5) cm and cross-sections for each of the following elements.Several plans have been delivered: -The four exterior walls of the minaret with scale 1/50; - The north and west exterior walls of the Mosque with scale 1/50; - The four interior walls looking over the inner court with scale 1/50; - The two east and south entrances with scale 1/25; -Horizontal and vertical plans and cross-sections of court elements (big and small ablution fountains) with scale 1/20; -Horizontal and vertical plans and cross-sections of prayer's hall elements (Chair, Balcony, Grave and Mihrab) with a scale of 1/20; -Plans and cross-sections of ornamental elements (wooden, metallic and stone decorations, paintings and writings, scale 1/10 -1/20; 2. 3D vector models for each element (DXF format) obtained from photo-pairs by stereoscopic restitution or through multi-image photogrammetry; 3. Rectified images and photoplans of the corresponding topographic plans for the inner court with scale 1/50; 4. Topographic plans, including planimetric and elevation details for the Copula and modules of vaults of prayer's hall and the three corridors showing contour lines, distortions, dimensions of plan elements, heights and parameters, with a scale 1/25.
A set of geodetic points around and inside the Mosque has been provided to the Committee by the contractor in order to guaranty the accurate referencing of the project.
The contractor was asked to carry out metric calibrated cameras for the recording and to provide calibration certificates.
The following accuracy was required for all the produced plans: -If M is the denominator of the scale, then the required accuracy had to be (0.3mm *M); -If d is the contour interval, the required accuracy had to be (1/3*d).Four control points were required in the specifications for each couple of adjacent photographs.The scale of the photos was asked to be approx.1/4 of the scale of the plans to be produced.The ratio-base (B/D) was to be within (0.6 -1.4) except in cases where conditions do not allow that.For stereo-pairs, 60% of overlap was mandatory.The scanning accuracy of analogues images was asked to be not less than 600 dpi for magnified positives and not less than 1000 dpi for the scanning of original negatives.The contractor delivered also both paper copies and digital files (DXF format) for topographic plans and BMP format for photoplans.

Technical report
The photogrammetric documentation of the Great Mosque was produced by the company Geodézie CS (Czech republic) in 1999 and 2000.We remind the reader that in 1999 only a few digital cameras with low resolution were available.The use of cameras in architectural photogrammetry was for a long time determined by the setup of expensive and specialised equipment called metric cameras.Depending on the restrictions due to the photogrammetric reconstruction process in former times, only metric cameras with known and constant parameters of interior orientation could be used.The images had to fulfil special cases for the image acquisition (e.g.stereo normal case) (Grussenmeyer et al., 2002).
The photogrammetric measurements of the Great Mosque were carried out by different techniques such as single image photogrammetry, stereophotogrammetry and analytical resection photogrammetry (called today multi-image photogrammetry).
The DVP system (Digital Video Plotter, Figure 6) was used for the stereophotogrammetry (Boulianne et al., 1994).The Rolleimetric CDW (Close Range Digital Workstation) was used to process 3D points from an array of at least 3 photographs taken from different positions, with no need to establish any element of stereoscopy between them (Heritage et al., 1998).

Image-based techniques
In the previous section, we have presented an overview of the documentation available from the past project (2000).Most of the techniques used in 1999 have changed to so-called imagebased techniques (Stylianidis et al., 2016).
In this section, we investigate about such solutions of 3D reconstruction of the Mosque based on the processing of the photogrammetric archives.We explore the possibilities of using these archives to generate dense point clouds using Photogrammetry and Structure from Motion methods (Murtiyoso & Grussenmeyer, 2017).Several control points, based on visible details of the monument, and recorded by total station during the 1999 campaign, have been used for the exterior orientation of the images.Agisoft PhotoScan and PhotoModeler software packages have been used to orient the selected images and finally to generate dense point clouds of the inner part of the Mosque and the minaret (Figure 14).First results show that an accurate virtual reconstruction basd on the archives is possible with the current tools and for the virtual reconstruction of this historic site.
In the following part, we present the results obtained with PhotoScan (Agisoft).

SFM processing with PhotoScan
In this part, we are trying to take advantage of Structure from Motion (SfM) methodology available in PhotoScan to generate dense point clouds of the interior walls and the minaret of the Great Mosque of Aleppo The pipeline of the processing is the following: Regarding to the inner walls of the mosque (the part treated in our research), it has been divided into: southern wall, northern wall, western wall and eastern wall.These walls are covered by 29 images that meet the 60% to 80% stereoscopic overlap.On the other hand, the minaret is covered by 17 images, which meet stereoscopic overlap.In Table 1, we find details related to the images and control data available for each wall and for the minaret of the mosque; The

Exterior orientation
Although Agisoft PhotoScan does not manipulate metric cameras (i.e. the fiducial marks are not measurable in PhotoScan) and therefore no rigorous internal orientation can be achieved, pre-calibrated interior orientation parameters can be fixed.To achieve the automatic interior orientation in PhotoScan, the focal length, pixel size and sensor size must be defined.In our project, three cameras models were taken into account according to these parameters.
Relative and absolute orientations of the images that cover the inner part and the minaret were performed successfully (Figure 14).Tie points automatically extracted by the software were used to achieve the relative orientation (Table 2).On the other hand, the absolute orientation was calculated using all available control points.A priori accuracy of the control points was set to 2 cm.These points were measured manually on the different images.
Figure 14.PhotoScan project showing the control points and the oriented archive images (the camera positions displayed for the vertical part in the figure correspond to the minaret).
To assess the geometric accuracy of the orientation, all the control points were recalculated as checkpoints.In the following table, the results of the comparison of the coordinates from the reprojected control points are presented: We should be able to improve the results by taking into account: • A better selection of control points as the contrast in some parts of the images are low and should therefore be avoided, as it may affect the manual measuring (with higher residuals) of these points; • The addition of more images for the minaret (Wild P32 images from the 2001 campaign).The bad results for the minaret are probably due to a poor configuration (the stereo-pairs were initially planned for stereo-restitution only).We can see in figure 14 a lack of oblique images for the minaret.Hence, the efficiency of the SfM approach is reduced for that part of the object.
These results should be improved by revising the camera calibration parameters and the interior orientation process, as PhotoScan is not able to manage fiducial poins.We plan to recompute the project in PhotoModeler, as this software package is able to manage fiducial points.

Processing of dense point clouds
Dense point clouds, for the inner walls and the minaret were automatically generated from oriented images with PhotoScan.There are some holes in the resulted point clouds (Figure 15), due to the poor quality of the black and white images (film scanned as described in §3.2) and insufficient overlap between some images.
To improve the final model and especially fill the holes, we applied the following method to complete the point clouds:

•
We first transformed existing CAD drawings from the initial documentation into shapefiles, and export the resulted shapefiles into point clouds (LAS format, Figure 16).This step was achieved with the ArcGIS software; • Finally, we merged all the point clouds in a unique point cloud (Figure 17) with CloudCompare.

DISCUSSION AND FUTURE WORKS
This project is a contribution to the CIPA Database for saving the Heritage of Syria (Silver et al., 2016).Taking into account two decades of cultural heritage documentation projects managed by the first author, the photogrammetric archives of the great Omayyad Mosque of Aleppo (Syria) can be considered as a reference.The aim of this documentation collected between 1999 and 2001 by the Engineering Unit of the University of Aleppo was to provide topographical maps and elevations, planimetric and altimetric details of the walls and entrances, of the dome and the minaret of the Great Mosque.The archives used the Aleppo project can be considered as a valuable data source required for 3D reconstruction of destroyed or damaged historical monuments.There are other interesting archives of the Aleppo Mosque recorded by Prof. G. Fangi (Italy) who showed very nice drawings in Fangi & Wasbeh (2013), and in Fangi (2015Fangi ( & 2016)).
Today the recording approaches have completely changed.The aim of the documentation is to deliver dense points clouds obtained from photogrammetric recording (terrestrial and more and more drone based) and/or from terrestrial laser scanning techniques.Terrestrial laser scanning techniques are today widely used for recording large and complex objects, sites and caves (Grussenmeyer et al., 2016).Dense point clouds processed from images are used as an alternative or complementary method to laser scanning.
In the second part of this paper, we showed that archives images can be considered as a valuable data source to process point clouds.Recent photogrammetry and SFM tools provide alternative low cost solutions to achieve such a point cloud based reconstruction, despite the limited number of images available for project recorded several decades ago.The reconstruction work of the Aleppo Great Mosque from the archives images is still in progress but the first results are very promising.Most of the images from past projects have been recorded with "metric" cameras with fiducial marks.But only a few recent software packages (such as PhotoModeler) are still able to take the fiducial marks into account in the photogrammetric orientation process.We plan to reprocess the project with this tool in order to improve the accuracy and include more images, especially for the minaret.

Figure 1 .
Figure 1.Great Mosque of Aleppo (before and after the war).The minaret has been destroyed in 2013.

Figure 2 .
Figure 2. Views from the top of the minaret towards the inner court of the Mosque and the citadel of Aleppo (Photos from P. Grussenmeyer, April 2000).

Figure 4 .
Figure 4. Rollei 6006 image with reseau crosses.The crosses were used both to check the deformation of the film and as fiducial marks A Photoscan TD photogrammetric scanner from Intergraph was used to accurately digitize the black and white or colour slides from the film-based cameras at a resolution of 21 microns.

Figure 5
Figure5presents an overview of the photogrammetric workflows for image acquisition and image processing in architectural photogrammetry in use two decades ago(Grussenmeyer et al., 2002).The classical photographic cameras have their advantages in the unsurpassed quality of the film material and resolution and in the well-known acquisition technique.The process of analytical photogrammetry makes benefit of the knowledge and rich experiences of the human On the other hand the pure digital data flow had not image acquisition devices comparable to film based cameras in 1999.

Figure 6 .
Figure 6.DVP (Digital Video Plotter) System Additional images from the minaret of the Mosque have been recorded in 2001 by Master Students from INSA Strasbourg during training periods in Aleppo at the Engineering Unit of the University of Aleppo.Those images (colour slides) have been recorded with a Wild P32 metric camera (9cm x 6cm, 64mm focal length).The P32 films have been digitized with the Kodak CD Photo system(Hanke K., 2000).

Figure 7 .
Figure 7. Example of Wild P32 colour slide showing the minaret and the terrace on the top of the Mosque (2001).

Figure 8 .
Figure 8. Map of the Mosque and its surroundings: the dimension of the inner court is about 80 m x 50 m

Figure 13 .
Figure 13.Processing stages of photogrammetric archives of the inner part of the Great Mosque of Aleppo 1. Selection of appropriate images from the archives.Regarding to the inner walls of the mosque (the part treated in our research), it has been divided into: southern wall, northern wall, western wall and eastern wall.These walls are covered by 29 images that meet the 60% to 80% stereoscopic overlap.On the other hand, the minaret is covered by 17 images, which meet stereoscopic overlap.In Table1, we find details related to the images and control data available for each wall and for the minaret of the mosque;

Figure 15 .
Figure 15.Dense point clouds of the Interior part of the Mosque From top to down: South, North, West, East walls.

Table 2 .
Geometric accuracy based on the measurement of reprojected control pointsThe results for the North, West and East facades are good, almost acceptable for the South facade.But the results for the minaret are not acceptable at this stage and need more investigations.