BEFORE THE EARTHQUAKE: RECONSTRUCTION AND PRESENTATION OF LOST SITES AND BUILDINGS

The paper presents the results of a research aiming at proving the efficacy of perspective restitution from photos for the reconstruction of buildings and sites that disappeared in the past century. The case study is the town of Messina, sited at the northeastern corner of Sicily (Italy), facing the homonymous strait that divides Sicily from the Italian peninsula; in 1908 Messina and Reggio Calabria, at the opposite side of the strait, were levelled to the ground by a powerful earthquake, followed by a tsunami. No building or monument survived the destruction: a new town with the same name took the site of Messina. Except for few strongly reshaped buildings, the memory of the streets and monuments of the destroyed town is today kept by some photos, taken by the end of the XIX century by professional photographers. The experimentation on Messina aims at a twofold purpose: test the reliability and usability of perspective restitution for the reconstruction of lost buildings and urban sites; use photogrammetric tools for the recontextualization of architectural elements (cornices, capitals, portals, corbels) that once belonged to a destroyed building and are today exhibited in Museums. Such process allows the verification of the scale of the reconstructed building and opens new forms of presentation of Historical heritage: in this study panoramic images have been used to display the building ‘attached’ to the fragment during the visit at the museum and to present, on site, the building in its original location with the fragments repositioned. The destruction and fall of Messina strictly echoes the destiny of many other towns in Europe; the research aims therefore at showing the potentials of perspective restitution and the usability of this technique in many similar contexts.


Digital perspective restitution from period photos
The paper presents the results of a research aiming at proving the efficacy of perspective restitution from photos for the reconstruction of buildings and sites that disappeared in the past century. Digital drawing and modelling tools overcome many of the limitations that, in the past, made perspective restitution from photos inaccurate and of no practical use; traditional drawing tools made perspective restitution from photos a hard task, especially when vanishing points appeared at a great distance from the image frame; the output of perspective restitution were 1D or 2D objects, i.e. segments, angles, planar figures and profiles that were revolved onto the picture plane. Remote vanishing points are almost usual in perspective restitution from photos, especially for vertical lines; in order to mitigate the convergence of vertical lines, photographers used to place the camera on elevated points of views; when no elevated spot was available, photographers used special cameras equipped with mechanical sliders that allowed to adjust the distance between the lens and the camera and the rotation of the lens itself. 1 This is why the vanishing points of vertical lines are usually at a great distance from the image frame.
These arguments briefly illustrate the reasons why, in the past, perspective restitution was widely practiced on drawings but it was not considered an effective tool to extract features and measures from photos. Digital drawing tools enhance perspective restitution from photos and allow the 3D reconstruction of lost buildings and sites from a single image. Digital perspective restitution can reach even the furthest vanishing point; absolute orientation can be calculated in 3D space and 3D models can be directly built with the combination of 'interpretation' lines and planes 2 ; the correspondence between 3D reconstructions and the photographic image can be visually checked by posing a virtual camera on the center of projection. Digital representation makes perspective restitution a powerful tool for researchers aiming at the preservation and enrichment of the memory of buildings and sites that disappeared after the half of the XIX century, when professional photography began to be diffusely practiced. Since its origin photography privileged urban scenarios and monuments; that is why photographic archives are powerful sources for the documentation of buildings and urban contexts that were destroyed or reshaped, in the past 150 years, by war events, natural disasters, or urban renewal programs. Perspective restitution, like any photogrammetric technique, is capable of detecting only angles and proportions; in order to build measurable 3D models, absolute orientation must be performed. Such process can be developed when the photo displays extant objects; if such elements are surveyed with laser scanning or SfM photogrammetric tools, the quality of the intrinsic and absolute orientation of the image can be checked by the comparison between the point cloud or mesh of the object and its photographic image. When no extant objects appear in the image, period maps can provide angular data for intrinsic orientation and dimensional data for absolute orientation. In this study uses both sources were used for the reconstruction of the case studies. Digital restitution retrieves the location of the camera in the reconstructed scene; the restitution of this datum verifies or neglects the reconstructed perspective layout: if the center of projection results in an inaccessible spot (i.e. under the ground, behind an occlusion element, at an inaccessible elevation) the perspective restitution layout must be reconsidered.

Case studies
The town of Messina, sited at the north-eastern corner of Sicily (Italy), faces the homonymous strait that divides Sicily from the Italian peninsula. Messina and Reggio Calabria, at the opposite side of the strait, were levelled to the ground in 1908 by a powerful earthquake, followed by a tsunami. No building or monument escaped the destruction; the buildings that were partially ruined were levelled at a later stage because they were considered unrecoverable; a new town with the same name took the site of Messina. Except for a fistful of historic monuments (the Cathedral and a couple of churches) strongly reshaped by harsh restoration, the memory of the streets and monuments of the destroyed town is today entrusted to some photos, taken by the end of the XIX century by professional photographers. Photographers were attracted by the beauty of the town, located on a strip delimited by the mountains and the sea and enriched by several monuments. It must be kept into account that Messina had been the capital of the reign of Sicily for a long time and was therefore endowed with relevant churches and palaces. Many photos were taken after the earthquake; rescuers from many countries went to Messina and photographers documented the ruins of buildings and monuments. The research focuses two buildings: a church with an annexed monastery and a noble palace. The chosen case studies share a common feature: some architectural elements, that escaped the destruction, are today exhibited at the Regional Museum of Messina. The church of Santa Maria della Scala, built at the end of the XVII century ( fig. 1), was located at the edge of the historic town, on the side of a road that still connects the hills to the sea; the side of the church, attached to the front of the annexed monastery, faced this road. The fronts of the church and the monastery are documented by few period photos, taken before and after the earthquake. The façade and the side of the church were decorated by two marble portals that were recovered, restored and remounted in the courtyard of the Regional Museum of Messina ( fig. 2). The second case study is the palace of the Grano family, which was located in the inner part of the historic center and was presumably designed by Giorgio Calamech, a Roman architect trained by Bartolomeo Ammannati. Few period photos document the façade of the palace before and after the earthquake ( fig. 3). The main portal was decorated by rich corbels sculpted to portrait mythological figures; two of these corbels have been recovered and are today exhibited in the gardens of the Regional Museum of Messina ( fig. 4).    The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China

Background
In manuals of descriptive geometry and in books on photogrammetry perspective restitution is ignored or simply quoted; few studies focus this subject with adequate attention.
The most complete studies have been developed by Italian scholars: the first contemporary manual on perspective restitution was published at the end of the '70s; the manual illustrates different restitution methods with a simple, synthetic and clear text (Fano, 1979). At the end of the '90s along with the initial dissemination of PC, scholars in computer engineering tried to develop automated processes for line extraction, vanishing point identification, intrinsic and absolute orientation (Van den Heuel, 1998). In recent years further studies aimed at the automated extraction of 3D textured models from an image; software tools were announced and discussed in publications (Arslan, 2014), but no evidence of their use in further researches has resulted. Architects and scholars in architectural representation foresaw that digital representation tools could enhance perspective restitution. A book, published at the end of the '90s (Carpiceci, 1997), discusses in detail the technical specs of light, vision, cameras and photogrammetry, for the documentation of cultural heritage; perspective restitution is outlined in a few pages. A book entirely dedicated at this subject (Paris, 2000) discusses the historic evolution of perspective restitution and proposes a test on the use of digital drawing tools for the restitution of 2D figures from photographs. The combined use of descriptive geometry and digital representation leads to the proposal of a solution (Fallavolita et al., 2013) that uses 3D modelling tools for the intrinsic orientation of a photographic image and for the restitution of lost buildings. Recent studies use descriptive geometry to calculate photos orientation (Dzwierzynska, 2017), but no use of 3D modelling tools is proposed or discussed (Ramon-Constanti and Gomez, 2020).

Image orientation
The first step in perspective restitution is the calculation of the intrinsic (inner) orientation, i.e. the position of the center of projection with reference to the picture plane. This position is fixed by the line orthogonal to the picture plane through the center of projection; the point where this line intersects the picture plane is named 'principal' point; the distance between the principal point and the center of projection is the 'principal' distance.
In perspective restitution from photos the picture plane is the film or the sensor that records the image, the principal point is usually the barycenter of the rectangular image frame and the principal distance equals the focal length of the lens. Intrinsic orientation can be solved in different ways; one of the simplest solutions is provided by three couples of mutually orthogonal lines and their corresponding vanishing points. The lines through these vanishing points are the sides of a triangle; the orthocenter of this triangle is the principal point.
In 3D the center of projection results from the intersection of three 'calibration spheres' (fig. 5), having their centers on the middle points of the sides of the triangle and diameters equal to their size (Arslan, 2014). Extrinsic (absolute) orientation demands the measure of a segment; if the segment is oriented in space, the extrinsic orientation provides both the scale and the orientation of the perspective scheme. The steps that follows the solution of inner orientation is the 3D rotation of the perspective layout; the interpretation line through the vanishing point of vertical lines becomes parallel to z-axis; the inclination of the segment that measures the principle distance shows the inclination of the axis of the camera ( fig. 6).  When the orientation process has been performed any element displayed in the image can be reconstructed in 3D; the oriented photo, focusing the façade, displays the side of the church foreshortened; perspective restitution allowed the reconstruction of the linear distances between specific architectural features (cornices, intersection lines), measured on horizontal and vertical lines. The reconstructed segments were used to calculate the extrinsic orientation of a photo that shows the side of the church with the annexed monastery ( fig. 9). This process links the position and orientation of the second image to the first one; the architectural elements reconstructed from the second image enrich the 3D model reconstructed from the first image. The process can be repeated with additional images that are linked in sequence; the reconstruction of the fronts of the church and the annexed monastery was performed with three photos at all.

Restitution
Perspective reconstruction of architectural elements in 3D virtual space is not an automatic process and demands education in descriptive geometry and in history of architecture. The reconstruction of the corbels flanking the windows of Palazzo Grano serves as an example of the potentials of 3D perspective restitution ( fig. 10). Figure 10. Reconstruction of the profile of a corbel.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China The absolute orientation of the perspective model has been developed to make the façade of the palace parallel to XZ reference plane. The first step of the 3D reconstruction process aims at the detection of the vertical line at the intersection between the façade and the corbel; this line results from the intersection of the XZ plane of the façade with an interpretation plane detected by three points: i) the center of projection; ii) the vanishing point of vertical lines (remote, not visible in the image); iii) a point A', taken on the photographic image of the vertical line. The intersection between the interpretation plane and the picture plane should match the photographic image of the vertical line; if this event does not occur, the perspective scheme has to be reconsidered ( fig. 10a). The reconstructed line is the Z-Axis of a local reference coordinate system used for the corbel's reconstruction; the mutually orthogonal horizontal lines at the base of the corbel are respectively X and Y axis; the profile of the corbel therefore lies on YZ reference plane. Such plane is used as construction plane, i.e. the plane where curves are traced ( fig. 10b).
In perspective view the camera is set on the center of projection and its focal length is set equal to the principal distance. The opacity of the photographic image is reduced to make the background visible from the center of projection. This way lines and curves can be drawn along the profiles that appear in the image ( fig. 10c), while actually they are drawn on the construction plane; the process allows the use of all the available tools for representation, e.g. circles through three points ( fig. 10d). The reconstruction of the curved profile of the corbel shows another potential of digital perspective restitution; traditional perspective restitution of curved shapes was a really complex task, since it was performed through the reconstruction of the position of some points of the curves. A further opportunity provided by 'digital' perspective restitution is the visual control of the position of a specific element in the 3D scene, from the center of projection. The following step of the corbel's reconstruction process addressed the duplication of the profile, by translation along the X axis, to match the opposite vertical face of the corbel and thus encompass its volume. The construction plane for this operation is XZ (fig. 11a); the operator duplicates the profile observing the scene in perspective view from the center of projection, until the copy matches its photographic image ( fig. 11b). The width of the corbel is thus fixed ( fig. 11c) and this architectural element can be duplicated to the opposite edge of the window and to the similar windows at the first level of the facade ( fig. 12a). Duplication verifies the proposed reconstruction of this architectural element and of the window as a whole. The restitution process in 3D space is developed through a constant check of the correspondence between the image and the reconstructed 3D model. The wireframe restitution provides the geometric structure of the 3D reconstruction model ( fig.  12b). Identical architectural elements, e.g. windows, are obviously duplicated on the façade, by translation on the XZ plane. The duplication process is finally controlled to verify that the distance between two adjacent windows is invariant along the entire length of the façade. The knowledge of the basics of architectural design is obviously mandatory, since it prevents errors due to a misinterpretation of the structure of architectural artifacts. Architecture has often been combined to sculpture in the past; the chosen case studies point at a further potential of digital perspective restitution: the virtual reposition of the models of sculptural elements in their original location inside the reconstructed 3D scene.

From the museum to the reconstruction model
The balcony over the portal of Palazzo Grano was supported by three huge zoomorphic corbels. Two of them survived destruction ( fig. 13) and are today exhibited in the gardens of the Regional Museum of Messina (fig. 14). The corbels have been surveyed with SfM photogrammetric tools; a couple of laser scans allowed the data needed to scale and orient the photogrammetric model.  The rotated models of the corbels were finally placed in the 3D reconstruction model; initially the upper plane of the sculptures was aligned to the intrados of the balcony; the following task was the translation of the models along the X and Y axis in their original position. Photos show that the male figure (left in fig.  14a) occupied the center position of the arrangement and was flanked by two identical female figures. The only extant female sculpture could be therefore replicated by symmetry to reconstruct the entire decoration. The position of the three corbels was reconstructed with a workflow that echoes what was done for the reconstruction of the corbels of the windows. The translation of the models has been controlled in perspective view from the center of projection of the photographic image. The good match between the corbels and the 3D repositioned models of the corbels verifies the orientation and the scaling of the perspective reconstruction framework ( fig. 15). The same process was used to reposition the portals of the church of Santa Maria della Scala in the 3D reconstruction model and the correspondence of the models and their photographic image validated the reconstruction process ( fig.  16). The reconstruction of the façades of Palazzo Grano with the anthropomorphic corbels and the facades of the church Santa Maria della Scala with the marble portals suggests a further possibility for the presentation of these lost buildings, that performs an inverse reposition process that brings the buildings inside the museums. The alignment between exhibited elements and the reconstruction model allows the development of AR or VR tools.   Photos of the courtyard where the portals are exhibited were taken with 35mm lens and fish-eye lenses; fish-eye photos were taken with the aid of a Nodal Ninja bracket, fixed on a tripod. All photos were uploaded in the photogrammetric software and their relative position was calculated. Fish-eye photos were grouped and the group was classified as 'station' group, i.e. a group of photos taken from the same point of view. The entire photogrammetric model was oriented and scaled with reference to the laser scanning point cloud, and thus the coordinates of the spheres could be calculated.
In the modeling software a sphere was calculated and was mapped with the equirectangular image generated by the photogrammetric software 3 . This way the equirectangular image resulted aligned to the laser scanning point cloud. The following step is the alignment of the 3D reconstruction model to the exhibited fragment. It is nothing but the inverse of the process discussed above (fig. 17). The visitor at the museum, standing in front of the portal, could easily view this element in its original architectural context. The proposed solution works when the exhibition arrangement proposes a point of view that is quite similar to the original one.
3 Agisoft Metashape and McNeel Rhinoceros are the photogrammetric and modeling tools used in this research.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China Figure 16. The façade of Santa Maria della Scala in the courtyard of the Regional Museum of Messina.

CONCLUSIONS
The paper reported the results of a research aiming at evaluating the potentials of perspective restitution from photos performed with digital representation tools, for the reconstruction and presentation of lost buildings and sites and for the recontextualization of architectural fragments exhibited in museums. Equirectangular images have been used as a tool for the presentation of these reconstructions. Further developments should address the use of AR tools for the visualization of these reconstructions and for the presentation of the buildings in their original location in the present town.