A LAYERED-WEB INTERFACE BASED ON HBIM AND 360 ° PANORAMAS FOR HISTORICAL , MATERIAL AND GEOMETRIC ANALYSIS

The latest information technology developments in the field of digital heritage (DH) have enabled the creation of novel virtual experiences favouring information-sharing connected to the 3D digital reconstruction of historical and existing buildings. In recent years, Building Information Modelling for historic buildings (HBIM) projects are the most applied methods to transmit the richness of built heritage from both the geometrical and informative points of view, but they are not always adequate to ensure a simple reading of different type of information for not-expert users. For this reason, further investigations were performed, following a novel SCAN-to-BIM process based on novel Grades of Generation (GOG 9 and 10) in order to digitally represent one of the most complex structure of the Basilica of Sant’Ambrogio in Milan. It is composed of arches supported by pillars, characterised by a double-wave curve, both on its vertical and horizontal axis. The information coming from the previous studies, such as historical documentation, as-found drawings, building archaeology analysis (materials and brick surface texture) and decay data (crack patterns, material discontinuity), was integrated with a novel web-immersive solution able to display 360° pictures, video and HBIM simultaneously. A detailed comparison of the 360 multimedia data of the current arrangement with the one of the HBIM historical phases allow experts and not-experts to analyse the transformation of the arch structures in a new immersive environment with different devices such as laptops, mobile phones and latest virtual and augmented reality (VR-AR) headset.


INTRODUCTION
Information and Communication Technology (ICT) is quickly revolutionizing the field of Digital Cultural Heritage (DCH).In particular, Historic Building Information Modelling (HBIM) are gaining a lot of popularity since they can be properly used for documenting, archiving, preserving and visualising digital cultural heritage (Murphy, 2011).Moreover, HBIM based on detail 3D survey (laser scanning and digital photogrammetry) is enabling holistic BIM-based cloud where interdisciplinary disciplines such as restoration and preservation of historic buildings can improve their own digital workflow.Traditional digital process and 2D CAD drawings are updated with new type of 3D representation able to convert 3D scans, also known as point clouds, in 'informative' models.This change of methods is called SCAN-to-BIM process, which allows professionals to accurately create digital models that follow the detected reality from a morphological and typological points of view with high levels of geometry (LOG) and information (LOI) (Brumana, 2018).Furthermore, thanks to a precise digital reconstruction of the detected artefact, it is possible to achieve digital workflows able to holistic transmit a different type of data and orient the model to different disciplinary sectors, bringing the traditional building management techniques to a new level of information-sharing (Banfi, 2016).On the other hand, most of the time, all these types of data require advanced skills and knowledge of a wide range of postprocessing and modelling application such as Autodesk Recap, Autodesk Revit, MC Neel Rhinoceros and others.Restorers, structural engineers and not-expert modelling users need innovative solutions that avoid the direct use of BIM applications and easily display specific information for specific areas of expertise and interest.In fact, a SCAN-to-BIM method based on new modelling requirements led the generation of one of the most complex historic building in Milan: the Basilica of Sant'Ambrogio in Milan.From a morphological point of view, thanks to the proper use of novel grades of generation (GOG) and accuracy (GOA), it has been possible to create each architectural and structural element of the building with a deviation value (between point clouds and model) of about 1/2 mm, favouring structural and decay analysis (Stanga, 2017).The approach required different application not easy to use for not-expert users and a BIM-based cloud that allowed the sharing of all the data produced during the generative process.In particular, the complex arch structure of the church required an ultra-detailed model able to show all the structural anomalies and the surface analysis.In particular, the arch structure is one of the hidden treasures of this well-known building.On the northern side, it is hidden by Bramante's Canonica, while on the southern side it is compromised by the chapels.For this research, the focus is on the northern side, since it is more accessible and visible.The structure is composed of four arches supported by pillars.Certainly, there were other two arches that were demolished during the aerial bombing of the Second World War.Its complex geometry, characterised by a doublewave curve, is probably the result of different construction phases and restorations.It was presumably built in the Middle Ages (around the 12th century) and the reason for its construction is still unknown (Fig 1).In order to share, mainly to not-BIM expert users, all the 'hidden' richness of this complex structure, further investigations on a new type of display were performed.

RELATED WORKS AND RESEARCH OBJECTIVES
In recent years, the research applied to digital heritage (DH) produced a wide variety of interesting system able to communicate a high quantity of information through the use of innovative digital model, greatly increasing the awareness of the intangible values of the buildings examined among all the users involved in the process (Della Torre, 2008).One of the main fields of application of HBIM beyond the restoration sector, is the mixed reality, in turn, composed of two different types: virtual reality (VR) and mixed reality (MR) (Sherman 2018).Both these types allow the creation of new environments and visualizations where digital objects co-exist and interact in real time.
One of the main advantages is the possibility of reusing HBIM models for the development of immersive environments where users can immerse themselves in the first person and interact with different types of content (Ioannides 2009).
The development of these environments, on the other hand, requires specific skills and knowledge of other types of software such as Unreal Engine and Unity (Schiavullo 2018).These applications, mainly created for the development of video games, led to the development of immersive environments oriented to the built heritage, thus facilitating a direct interaction of historical models with specific contents of historical-cultural interest.
The products of these applications are APPs that requires the use of latest generation hardware devices: headset and controllers such as Oculus Rift, Sony Playstation VR and Samsun gear VR.
Figure 2 summarizes the wide range of headset, controls and hardware types required by the most used VR devices in the field of Digital Cultural Heritage.For this reason, the following research was aimed to generate an immersive environment that did not require the use of these applications, software and hardware, trying to reduce the time of sharing information through a more sustainable and scalable system for not expert users.
The best solution was found in the development of a webservice system able to merge different data source such as HBIM and 360-degree panoramic pictures, creating a comparison between different historic phase and decay analysis.The main goal was to support professional in an easy-reading of articulated information and complex geometry of the arch structure of the church.Figure 3 summarizes the method applied to the case study of this research, relating the different data sources, the application used for each generative step and the related contents to the main research objectives.
Figure 3.The digital workflow applied to the research case study.
In particular, • The first goal was to intercept the morphological features of the structure by different 3D survey technique in order to give to the HBIM generation the proper geometric bases.
• The second research goal was to accurately represent the model, applying new modelling requirements (GOGs & GOA) able to create an accurate digital model from different geometric primitives.
• The third research goal was to convert the digital model in a real HBIM with a high level of information (LOI).In order to improve the LOI, different historic and on-site analysis were performed and linked to the model, this way all the richness of the church was collected and digitalized.
• The fourth objective was to convert specific information into a web-service solution able to shows different model and historic information to not-BIM expert users.

Data collection and 3D survey
The survey of the Basilica of Sant'Ambrogio was carried out in two different phases.In the first phase, the main nave of the basilica and its four-side portico were surveyed while in the second phase the survey focused on the rear part of the apse and on the arch structures.Scans were registered with an average precision of ±3.0 mm by using chessboard targets measured with the total station and additional scan-to-scan correspondence (spherical targets).
A photogrammetric survey was carried out for each arch and pillar, side and front walls and arch intrados.
The main aim of this was to realize orthophotos at the scale 1:10.In order to process the data PhotoScan was used and a set of Ground Control Points (GCPs) and Check Points were derived from the laser scanning survey to constraint the reconstruction and verify the metric accuracy of the final results.
In particular, considering Check Points a RMSE of about ±0.0103m was obtained.The result depends on the fact that the Check Points were identified by choosing significant points on the point cloud.In this phase no targets were used.Those orthophotos were used as the basis for a detailed archaeology analysis of this portion of the building.
Figure 5.The orthophotos were used as the basis for geometric and building archaeology analysis.

Model generation: from geometric primitives to complex HBIM
Data collection and 3D survey provided different data sources such as historic documentation, accurate 3D scans and detailed orthophotos, to properly created an HBIM able to show the complexity from a morphological and typological point of view.Laser scanning and digital photogrammetry allowed the identification of complex geometries and lay the foundations for the application of modelling requirements for SCAN-to-HBIM process.A grade of accuracy (GOA) of 1/2 mm was achieved thanks to the application of different grade of generation (GOG) (Banfi 2017).
In particular, in order to create a proper model able to show all the detected complex shapes, the generative key was applied GOG 9 and GOG 10 at the same time.GOG 9 is based on the generation of NURBS from 3D wireframe.It is a modelling procedure that summarises the known slicing technique.The traditional process of generating 2D CAD models and drawings is essentially based on the manual extraction of cross-sections from point clouds.The sections, usually used for technical drawings, are used as a generative basis for three-dimensional modelling in the digital space.GOG 9, instead, bypassed 2D drawings and permitted the generation of complex geometric models through the automatic and semi-automatic extraction of geometric primitives point clouds.They are controlled by a closed exterior edge and internal section curves.In BIM application it was possible to realise a 3D object with the related thickness of the strucutres selecting the NURBS surface previously produced.
The main parts of the structure required also the application of GOG 10 (NURBS from a set of point) able to intercept the geometrical and structural anomalies with a double curvature inherited from a millenary historical heritage.The generation of internal curves able to intercept the irregular internal shapes of the complex elements was substituted with 2 modelling requirements.The tested method allowed the generation of historic elements interpolating a closed exterior edge (i) and a set of internal control points (ii).After the NURBS generation, in BIM application it was possible to realise a 3D object with the related thickness selecting the surface previously produced.This procedure allowed to bridge the modelling gap in Autodesk Revit (based on GOG1 to GOG 8) and to create a NURBS model able to accurately follow the 3D scans (Fig 6).
Figure 6.The generative SCAN-to-HBIM process based on GOG 9 and GOG 10.Thanks to new modeling requirements it has been possible to achieve high LOG and LOI.

HBIM mapping: the proper connection of different information and data to the historic model
The study of the arch structure is essential for the understanding of the constructive and historical phases of the church.In particular, it is crucial to shed further light on the Romanesque building site, like the materials that were used (and reused), the bricks surface texture, and on the restorations that were carried out in the 19 th and 20 th centuries.The 11 th -13 th centuries are the period of the "rifabbrica romanica" (Romanesque reconstruction), when the church changed its appearance thanks to the construction of the vaulted system, which replaced the former wooden roof (Landriani 1889).However, few literatures focused on the arch structure that were built almost at the same time.Since the arch structure is relevant for the Romanesque phase, and it is still not promoted inside the basilica guided tour, it is important to easily show on the HBIM the historical information and construction data related to it.Different kind of data were linked to the HBIM, such as the photogrammetry survey and orthophotos, historical drawings and pictures, architects' reports and measurements, building archaeology stratigraphic units datasheets and Harris matrix, materials properties and external links to the Sant'Ambrogio's cloud.The simple mapping of the HBIM with the orthopothos is one of the main contributions to the knowledge of the arch structure.It allows to show the detailed building archaeology analysis that helps the understanding of the basilica and the different construction phases of the arch structure (Fig 7).This is not part of a homogeneous design plan.First the pillars were built, not in structural continuity with the basilica wall, then, the arches in between the pillars were realized.It is clearly visible from two vertical signs that start at the impost of the arches and continue to the top of the pillars Figure 7. Building archaeology analysis applied to arch n. 5.
The stratigraphic analysis datasheets include many information regarding each stratigraphic unit (masonry, plaster, etc.) that were detected during the survey (stratigraphic relationship among the units, material characteristics, on-site observations).
Thanks to them, it is possible to know the different kind of bricks that were used.Most of them came from the former roman construction, like the thin roof tiles or the "suspensura" bricks, which were part of the round piers that supported the floor of roman baths.The "motivo a mattonetti" is another peculiar characteristic of the Romanesque building site, which involved the reuse of roman bricks by placing them on rowlock lines or on herringbone patterns (Fieni 2004, Fiorani 2008).The main used material is the brick, but some stones are adopted as orthostates and arch imposts.The mortar joints were instead probably made in the 20 century, since it seems that the mortar is cement.The addition of the material properties in the HBIM allows to focus on the different bricks surface texture.Many bricks have scratches on their surfaces: some of them are straight, some are 45° straight, others have a "chevron" pattern (Fig. 8).
The scratches were usually realized before baking the bricks and it is common to find them on the Romanesque buildings.
The literature offers different opinions about their meaning: they could be used to help the plaster to easily grip on the masonry, or they could be a surface finishing and have an aesthetical value (Fiorani 2008, Redi 2000).Some of them can perhaps be referred to the 19 th or 20 th century works, as "marks" made by the restorers to distinguish with a different surface texture the bricks that they added from the older ones.In fact, during the 1850s-overall-church restoration, the single-lancet window of the side wall of the basilica (at the level of the matronei) were enlarged in order to improve its interior lightening.That is why, it is possible to find many misalignments between the mortar joints of the bricks of the wall and the bricks used to make the splayed jambs of the single-lancet windows.The author is unknown, but probably the drawing is related to the 19 th century-new-design project of the northern side of the church.These documents, together with the 2D geometrical survey realized for this research, were collected and added to the HBIM in order to create a first database of pictures of the arch structure.
The HBIM of the historical phases of the arch structure embeds all of this information in a 3D environment.It represents the 4 historical phases that were detected.The starting point is the basilica after its Romanesque re-construction.At one point it presented pillars alternatively with rectangular (the bigger ones) or polygonal (the smallest ones) shapes (first phase).This is also one of the reasons of the peculiar double-wave curve of the structure.In fact, the need of joining together smaller and bigger pillars could have resulted into a horizontal and vertical curvature.The first reconstruction of this arrangement is made by De Dartein, who drew the plan of the basilica as it would have appeared in the 12 th century.His hypothesis was later corroborated by Ferdinando Reggiori, the architect in charge of the 20 th -century restoration.He had the opportunity to measure and report in his drawings 2 of the former polygonal pillars.These drawings, together with former geometrical survey data stored in the S. Ambrogio archive, were digitalized and attached to the HBIM in order to georeferenced the historical information (Fig. 9).
Figure 9. Single-lancet windows construction history.
In the second phase, all the pillars had the rectangular shape, and later the arches were built in between them (third phase).In fact, it is possible to see that the arches of the structure cover the Romanesque little arches decoration of the walls.It is not possible to certainly state when these two phases were carried out.In any case, it is quite sure that at one time the pillars visible -they were not hidden behind other constructionsbecause a mural painting was found on the pillar of the arch.3 on a lower level of the current first floor of the Canonica.In a picture took by Reggiori it is possible to see 3 steps that were found during the restoration of the church on the pillars between arch n. 5 and 6 (the picture can be found in the Basilica di S. Ambrogio archive).Both the mural painting and the steps are witnesses of former arrangements of the northern side of the basilica.
The last phase (fourth phase) is the construction of the northern chapels and the Canonica by Donato Bramante (end of 15 th century).Bramante decided to build the wall of the chapel 1 meters away from the pillars.The reason of the construction of this structure is still unknown, it is highly likely that it is a kind of buttresses system of the church.
In fact, the church suffered of some structural problems during its life cycle as the out-of-plomb of the walls demonstrates (Fig. 10).
During this research three main hypothesis were made.The first one relates the construction of the arch structure to the need of improving the structural behaviour of the church after the collapse of the first vault of the Nave around 1193-94 (Porter 1917, Ambrosioni 2003).
The second one is related again to the vaults and considers the arch structure as an additional support to the whole vaulted system, together with the matronei (Reggiori 1966).
The third one is related to the possibility that the arch structure helps the structural behaviour of the church also in case of an earthquake: the pillars counteract the vaults thrust, while the arches can better resist to the horizontal loads.
Figure 10.Out-of-plomb of the basilica walls.

The development of layered web-interface based on 360° panoramas for historical, material and geometric analysis
In recent years different types of photographs visualization have emerged.In particular, 360°cameras gave the possibility to create virtual reality, immersive photography and Virtual Tour (Bonacini, 2015;Banfi et al., 2018).The acquisition of the spherical images was carried out by using the RICOH THETA V.The camera acquires two images acquired with a fisheye lens that are stitched together to generate a 460°image (equirectangular projection).The acquired images were used to texture the HBIM model.As described in the previous paragraph, HBIM generation required the integrated use of GOG9 and GOG10 in order to create a complex NURBS model able to follow the point clouds with a grade of accuracy of about 1 mm.As a result, the SCAN-to-BIM model of the arch structure allowed to map the model with the orthophoto threedimensionally.The problem of how to represent the texture in NURBS surface has been overcome by means of uv mapping.U and V are the coordinates of the texture corresponding to X and Y.A mapping channel holds a set of texture-mapping parameters.Each mapping channel was identified by a number.
An object can have any number of channels and therefore can hold any number of texture mapping types.Any time an image was applied as a material to the model, uv texture mapping has been used.Textures in materials can be assigned a channel number.When the textures are applied to the object, the texture is attached by using the matching channel number of the object.
Figure 11 shows the method applied to the HBIM.It is composed of four step procedure: • The definition of each picture edge for each trimmed NURBS surface.
• The definition of a U and V NURBS subdivisions and directions for each NURBS surface.
• The assignment of the 3D texture to each NURBS surface.
• The comparison of the mapped NURBS surface with the point cloud in order to check the quality of the "matching" phase.The last step been the generation of a virtual tour from a set of panoramas.
In to support a simple reading by not-expert users, a freeweb tool has been used to processes the panoramas and generate an easy-to-use web application.
The first panorama presents the current arrangement of the arch structure and the orthophotos of each intrados arch and pillar have been attached to the model.The second panorama is related to the study that was carried out by experts in order to understand the different construction phases of the structure through the building archaeology analysis.
Below, the description of some of the main stratigraphic units.The depiction of each unit is based on the Italian common standards of representation (Boato, 2008).The wall that separates matronei and S. Ambrogio's room is the USM 201 unit, which is characterised by bricks laid with the shorter or the longer edges visible without a homogeneous arrangement.
The joints are made of cement mortar with bucket handle profile: they were probably done during the 20 th -century restoration.The USM 201 is the stratigraphic unit that includes all the peculiar features of the Romanesque building site, as explained before (reuse of roman bricks, different bricks textures, etc.).
The "negative" unit USN 218 identified on each span is related to the widening of the single-lancet windows during the 19 thcentury restoration, has already mentioned.It "cuts" the USM 201 and is "filled" with USM 219, in fact the bricks of the USM 219 are not aligned with the ones of the USM 201.Another interesting "negative" unit is the USN 216 that represent the breach between the arches and the pillars, which means that the two elements belong to different construction phases.
Many other stratigraphic units were recognized on each part of the structure.
It is really curios that the intrados of each arch has many "negative" and "positive" units, which means that those part were rehashed and restored several times.Each stratigraphic unit were record in specific datasheet.

CONCLUSION AND FUTURE DEVELOPMENTS
Digital technologies are changing the approach to the Built Heritage, clearly showing the new paradigm of complexity.As a consequence, architects, restorers and experts are facing new challenges in their work.
The growing need of 3D representations led them to differently approach to the study of the Built Heritage, moving away from the traditional 2D drawings and thinking about the 3 dimensions of the building.
At the same time their way of design restoration plans and interventions is affected by these changes.
Undoubtedly, the interdisciplinary cooperation between BIM operators and Cultural Heritage experts is a fruitful one because it allows to reach high level of analysis, but at the same time it is necessary to share the same "working" languages and vocabularies.
On the other hands, each expert has its own area of expertise, that is why the cooperation has to be developed through jointapproaches that allow experts to understand the different contributions of the study.This paper offers some considerations about this discussion and contributes to the focus on different ways to make BIM technologies available to experts (by making them able to do their specific analysis) and not-experts (tourist-oriented purposes) (Fig. 12).

Figure 1 .
Figure 1.Section, plan and pictures of the northern arch structure of Basilica of Sant'Ambrogio in Milan.

Figure 2 .
Figure 2. The different type of devices, headset and controls for VR projects.
During the first epoch, a geodetic network was materialized and measured with a first order Total Station Leica TS30.The network is made of 8 stations and after least squares adjustment the average precision on benchmarks was of about ±1.0 mm.In the second stage a new set of stations were added to the original network.Repositioning was performed by resection using as benchmarks a set of retro reflective targets measured in the first epoch.The scan acquisition was carried out with Faro Focus 3D.In particular, to acquire the arch structure the instrument was placed in the room close to S. Ambrogio's room in order to record the arch n.3, in S. Ambrogio's room for the arches n. 4, 5 and 6, and finally in the narrow corridor for the arch n. 7 for a total number of 5 scans (Fig4).

Figure 4 .
Figure 4. Point clouds data of the Basilica of Sant'Ambrogio.

Figure 12 .
Figure 12.Information sharing among experts thanks to different type of visualizations and devices.