THE VIRTUAL HISTORICAL RECONSTRUCTION OF THE CERCHIA DEI NAVIGLI OF MILAN: FROM HISTORICAL ARCHIVES, 3D SURVEY AND HBIM TO THE VIRTUAL VISUAL STORYTELLING

In the last decades, thanks to the development of eXtended Reality (XR) development platforms, research studies have envisaged new possibilities for disseminating tangible and intangible values of the past through historical virtual reconstructions. More recently, experimentation in the field of virtual archaeology has led to the development of interactive experiences based on advanced virtual and augmented reality (VR-AR) technologies. In this context, the authors intend to support the transmissibility of the historical and cultural background through validated and sustainable workflow-oriented to create virtual historical reconstruction using different sources such as 3D survey (laser scanning and digital photogrammetry), Historic Building Information Modelling (HBIM) models and historical reports. The XR project’s development included reconstructing the main historical sections and water basins of the Cerchia dei Navigli in Milan: the Conca dell’Incoronata, the Conca di Viarenna and Via Senato. The main objective is to achieve realistic and suggestive Virtual Visual Storytelling (VVS), recreating the atmospheres and scenes of everyday life when the waterways were still present. The result is engaging content on an emotional level, which leaves the visitor with dynamic memory and the feeling of reliving lost moments.


General context
In recent years, the process of preservation of heritage buildings and archaeological sites has benefited from the use of the latest generation techniques and tools such as 3D survey, historic Building Information Modelling (HBIM) and eXtended Reality (XR) (Jung et al., 2021;Pybus et al., 2019). The 3D survey characterised by the integrated use of cutting-edge tools such as the laser scanner, total station, and digital photogrammetry (terrestrial and aerial) made it possible to collect, catalogue and use a large amount of data (Lerma et al., 2010). The main inputs such as point clouds, orthophotos and highresolution textures were used for both the digital reconstruction process (as-found) and the historical scenarios (XR projects) of the main archaeological sites and key points of the Cerchia dei Navigli in Milan (XII century), the latter almost completely disappeared both in reality and in the memories of the European community. For those reasons, given the historical and cultural importance of those sites, the following research shows how an integrated process of 3D survey, digital reconstruction, in-depth analysis of historical archives, HBIM for archaeological sites, XR reconstruction and 3D animations allows recreating, archiving, and reliving the key nodes of one of the most important water networks in European history, supporting new forms of simulation, sharing, digital communication with a greater awareness of the tangible and intangible values of this precious historical cultural assets in the process of disappearance.

State-of-the-art: HBIM to Virtual and Augmented Reality (VR-AR)
Over the past five years, many restorations and rehabilitation processes have used the latest technologies such as 3D surveying and Building Information Modelling (BIM), leading to an epochal transition from the first to the second digital era. The integration of the 3D survey with BIM has made it possible to define scientific methods capable of digitising the surveyed artefacts, expanding their information value during the life cycle of the building. In the context, BIM for heritage buildings (HBIM), infrastructures, archaeological sites, and monuments, has turned out to be a tool capable of improving the quality of the works of the professionals involved and improving the transmissibility of information such as the features of the materials (physical and mechanical), the wall stratigraphy and historical phases. Consequently, the transition from an early digital era characterised by CAD vector design to a digital era strongly influenced by informative digital models required new skills and knowledge in disciplinary sectors other than restoration, geomatics, and architectural representation. In addition to them, architectural representation has been influenced by Information Technology (IT) developments and by the advent of new tools capable of improving and exponentially transmitting informative digital representations with a high Level Of Detail (LOD), Level of Information (LOI) and accuracy. The main tools that led to this change are modelling applications, BIM platforms, and methods to transform simple points from the 3D survey into parametric objects (scan-to-BIM process). One of the main differences between 2D CAD vector representation and digital models consists of transmitting, in addition to a heterogeneous multitude of information, models capable of supporting the life cycle of the building with different configurations. In fact, one of the main advantages of HBIM is that it allows professionals to follow the time factor through different 3D configurations, passing from as-found models to advanced models such as asdesigned and as-built projects.
In this context and from a strictly methodological point of view, the historical and project phases, enucleated information in BIM projects, schedules and databases have allowed users to go beyond simple project management, restoration, analysis, plant engineering, construction site and maintenance over time (Volk, et al., 2014). In addition, even more, advanced projects have made it possible to share HBIM projects articulated in digital form based on web platforms. Thanks to cloud-based BIM platforms, all the actors involved can interact around a shared model and reach the information entered in it in a more sustainable form. Consequently, BIM has almost wholly supplanted manual representation, leading construction professionals to increase their knowledge in IT. On the other hand, in addition to the countless advantages found through the integrated use of survey instruments (laser scanning, total station and digital photogrammetry) with HBIM, various fields of application have not yet been fully investigated.
As is well known, the development of Virtual and Augmented Reality (VR-AR) tools has allowed a further step forward in digitisation. AR-VR are tools capable of increasing the sharing, immersion, and interactivity of the models, which strongly depend on a high level of knowledge of computer techniques and latest generation tools such as the VR headset. Many disciplines and fields of application today involve the use of these new technologies. Interesting studies and application uses have shown how VR and AR can be decisive for different objectives. It is well known that VR and AR are two of the fastest-growing sectors in IT. As big brands look for ways to incorporate VR into their daily offerings, small start-ups continue to make the fundamental advances that can change people's (Alizadehsalehi et al., 2020;Atsiz, 2021). One of the best daily uses of VR-AR technology currently in use can be understood through the virtual positioning of products before their purchase. Many companies, such as Amazon and its AR View technology, allow users to place products in your home in virtual visions even before they are purchased. The product can be manipulated, rotated, and repositioned as desired before being fixed in place for viewing from different angles. Furthermore, in the medical field the thinking behind many companies that are developing applications and contents that exploit the power of VR-AR is to create training simulations for trainee doctors and surgeons, allowing them to enter a virtual operating room and follow the procedures step by step, discovering expert disciplines and techniques. Outside the fields of science and technology, VR-AR are making its way into the workplace through the training of employees and students. In this context, the XR development platforms can be key factors to transmit the tangible and intangible values of historical artefacts that may not be handed down from generation to generation over the centuries.

State-of-the-art: serious games for interactive virtual reconstruction
This study looks at the opportunities created by new XR technologies introduced recently in the context of the virtual reconstruction of heritage sites for entertainment, learning and new levels of information sharing of digital models. The paradigm of interactivity of immersive 3D environments has found the main focus of the proposed scientific approach. In recent years, the scan-to-BIM process and HBIM have stimulated a lot of research in computer vision (Bosché et al, 2015;Chen et al., 2020). 3D modelling, 3D survey and digital photogrammetry proved to be the appropriate basis for the creation and simulation of future or past virtual environments. In this context, serious games, or games that take advantage of the playful aspect of the game to facilitate the objectives of education and instruction, are "serious games" that aim at education through fun. There are different types of serious games, from the simplest to the most complex ones, that use technologies such as VR and AR to reproduce situations in which the player can learn by experimenting (Hammady et al., 2021;Huhtamo, 2013;Helander, 2014;Sylaiou et al., 2010). In this case it is learning through a virtual simulation where the interactivity paradigm is crucial to achieving a series of requirements unlike traditional video games: • learning by doing active, experiential learning achieves qualitatively better results than passive learning; • convey contents that, through classical training, would be much more difficult (and tedious) to learn; • active involvement of the user who therefore becomes the protagonist of the training experience; • the possibility of recreating a protected environment in which the user has the opportunity to discover and get involved without being afraid of making mistakes but can actually learn from their mistakes; • development flexibility aimed at different types of users and devices; • Freedom of implementation -the use of open XR platforms allows you to create scenes which in turn can be implemented both from a geometric point of view and from a content point of view.
Based on these requirements and many other advantages that AR and VR serious games bring, we can therefore say that they are suitable tools for educating, training, and informing in an incisive and dynamic as well as pleasant and stimulating way (Alizadehsalehi et al., 2020). As is well known, on the one hand, video games are not mainly used for educational purposes because there is no interest on the part of the manufacturing companies. On the other hand, the properties reported above, of which the development of multi-tasking thinking (ability to consider several elements simultaneously to reach a goal), the development of critical thinking and the speeding up of thinking and learning processes are clear examples, they have only recently been discovered, or perhaps it would be better to say accepted, by the scientific community. For this reason, this study seeks to settle a process capable of increasing the "utility'" of HBIM models through new forms of humancomputer interaction. Through a faithful and more reliable historical virtual reconstruction, the authors propose three XR projects with the main objective of improve the historical and cultural awareness of a heritage that in the future years will be the subject of urban redevelopment of primary importance not only for the older generations but also for future ones.

Research objectives
The digitisation process used different types of the latest generation digital tools intending to research informative digital worlds capable of interacting with the end-user (expert, virtual tourist, and students). The integration of in-depth analysis of historical material (treatises, reports, archives, historical photographs) and digital reconstruction through a scan-to-HBIM-to-XR method has allowed authors to pass from a static digital representation to historical interactive virtual reconstructions of taking life based on user inputs and share different types of content. The XR project's development included the reconstruction of the main historical sections of the city of Milan: The Conca dell'Incoronata, the Conca di Viarenna and Via Senato. Starting from a laser scanner survey, it was possible to start a detailed reconstruction of the archaeological sites and gradually reconstruct the three case studies through reliable historical sources and verified geometric data.
Thanks to a scan-to-HBIM-to-XR process, the article will show how the four phases of data collection (i), model generation (ii), information mapping (iii) and information sharing (iv) were articulated for the research case studies to obtain a serious game able to share different types of contents. Visual Programming Languages (VPL) and gaming software have been oriented to XR, from VR, through AR to the most innovative forms of 3D animation and digital interaction. In particular, it will be shown how through articulated Virtual Visual Storytelling (VVS), the Cerchia dei Navigli can come back to life and interact with the user through new virtual information points and interfaces in the first and third person. Furthermore, the serious game was oriented towards opensource devices and platforms to promote dissemination and communication for different devices such as mobile phones, PCs, and VR headsets (Figure 1).

Figure.1
The research case studies in relation to the proposed digitisation process and multiple devices and XR environment.

HISTORICAL AND CULTURAL CONTEXT: THE CASE STUDIES OF LA CERCHIA DEI NAVIGLI IN MILAN
Since Roman times, Milan was built and organised by a large network of canals, which favoured trade, irrigation of the fields and the main movements by water until the second half of the 20th century. The construction of the entire infrastructure lasted from the twelfth to the nineteenth century. The Cerchia dei Navigli was the "hinge" of the Milanese city that allowed the system to function as a whole (Cassi Ramelli, 1971). Built as a defensive moat of the medieval walls of Milan starting from 1156 and transformed into the Naviglio in its south-eastern part between 1387 and 1496 thanks to the canalisation and expansion works commissioned by the Visconti and the Sforza, the Cerchia dei Navigli was the connection of the main waterways of the region (Figure 2). The only part of the Cerchia dei Navigli that remained a simple non-navigable moat was that towards the Castello Sforzesco, or the stretch located to the north-west: its function remained in fact that of bringing water to the moat of the castle. The two branches of the Cerchia dei Navigli that headed towards the Castello Sforzesco were called Naviglio di San Gerolamo (the one that descended from the southern vertex of the fortification along via Carducci) and Naviglio Morto (the one that descended from its northern vertex along via Pontaccio). The most important use of the network of canals was intended for constructing the cathedral of Milan, where the canals were used for the transport of marble for the construction of the building. In the Renaissance period, Leonardo Da Vinci was entrusted with the design and engineering of the main basins, which made it possible to connect the sections of the canals at different levels. The canals were buried for urban, hygienic, and environmental reasons at the turn of the nineteenth and twentieth centuries.
On the other hand, the two most important basins, the Conca dell'Incoronta and the Conca di Viarenna are still present today, albeit limited to Leonardo's mechanisms and affected over the years by urban adjustments aimed at favouring car traffic (Figure 3).
After the construction of the Naviglio Martesana in 1463, it was natural to hypothesise the union of the latter with the inner circle of the Milanese Navigli near San Marco; the difference in height between the two waterways obliged to build a basin that could overcome this problem. Francesco Sforza had ordered to start the works in 1464 interrupted for several years, mainly because the technical problems due to the strong difference in height in the last part of the path. The problems were overcome at the end of the 15th century, with the intervention of Leonardo and the construction of a system of locks. It was 1482 when Ludovico il Moro assigned Leonardo (Pedretti 1981), who had just arrived in Milan, the task of studying a system that would make it possible to navigate between Lake Como and Milan. Leonardo had already designed a system of dams to solve the problem of height differences and made further improvements and modifications to the system in that occasion.
In 1496 the Cerchia dei Navigli was connected to the Naviglio della Martesana through the basins of Incoronata and S. Marco, thus becoming the centre of the Sistema dei Navigli Lombardi, which allowed the continuous navigation of the canal system.  For this reason, the Conca dell'Incoronata is considered the last testimony of the Naviglio Martesana in the north centre of Milan.
The sluice is basically a barrier that separates two parts of water with different levels whose function is to allow ships and boats between different levels. In addition to designing steps to The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China cushion the impact of water, Leonardo inserted a double door operated by a bolt, which allowed a better regulation of the pressure exerted by water on the doors of the sluice, operated continuously during the passage of boats. The Conca dell'Incoronata presents the last ancient bridge over the Naviglio and the remains of original navigation equipment with the recently restored wooden gates as those designed by Leonardo da Vinci, used for navigation and visible in the Codex Atlanticus and the "garitta" in bricks ( Figure 5). The Viarenna basin also worked as a boat elevator and had the task of making the ships go up or down from the small lake of Santo Stefano to the Naviglio (Malara, 2008). The large blocks of marble arrived at the final destination in the city, the small lake of Sant'Eustorgio, and had to arrive at the Duomo building site; the problems connected with the different heights of the canal system did not facilitate the transport and already in 1939, the engineers of the Duomo were working to solve the passage between the Navigli and the Fossa interna and had built the first basin. It was built in bricks and wood and finally given as a gift by Ludovico il Moro in 1497 to the Veneranda Fabbrica del Duomo with charges and privileges, including the maintenance of all the canals, together with the possibility of collecting duties except for the marble blocks used for the construction of the Duomo, which were instead marked with the initials "AUF" (Ad Usum Fabricae). The basin represented the obligatory passage for the transportation of the Candoglia marbles to the Veneranda Fabbrica del Duomo through the small lake of Santo Stefano. When the Spanish arrived in Milan and completely revised the system of fortifications, they commissioned the Veneranda Fabbrica to move the basin to its current location. This took place between 1551 and 1558 with the design of engineers Cristoforo Lombardo and Vincenzo Seregni. Demolished in 1555, a wooden platform of its base remained as the fifteenthcentury tombstones in the aedicule attested to the sixteenthcentury Conca (Bruschetti, 1821). Later, from 1885, various urban transformations took place. In the second half of the nineteenth century, the increase in population caused the city's sanitary problems, making the conditions of the waterways critical. In 1889 the Beruto Plan planned the covering of the canals and the total closure of the canal connecting the Conca with the Darsena, completely isolating the structure. Referring to the principle of "salus publica suprema lex" Beruto supported a coverage of the inner circle and the most famous waterways such as Redefossi, Seveso, Vettabbia and Borgognone. On March 3, 1928, the Ministry of Public Works is asked for permission to cover the "internal ditch". In March 1929, work began on the closure which lasted about a year. Thanks to the historical study of the two basins, it was possible to understand the construction techniques. The main elements are: (i) the filling tank, whose bottom is located at the same altitude as the main watercourse, (ii) two barriers called gates that allow you to manage the water level, (iii) the mandrake that allows passengers to get off boats and finally (iv) the tank also called spillway necessary for filling operations. The basin's interior is often covered in brick to avoid damage to the infrastructure by the boats that use it, while the doors were made of wood ( Figure 6). The first excavation of the circle of the Navigli was nothing more than a defensive ditch dug around 1140-1150 (Malara and Milanese, 1989) to obtain the necessary land to build defensive bastions to protect the town, over time several times the ditch and walls were the one widened and the other reinforced until 1329 when yet another enlargement of the ditch and strengthening of the walls created the circle of the Navigli making Milan a rich city for commercial water transport. But the stretches of the inner circle of Via San Damiano and Via Senato, for example, was never a waterway for the transit of barges and materials, but more a navigable part to access homes, palaces characterised by numerous gardens and frequented by the nobility (Figure 7). A bifurcation formed a basin of navigation and a derivation that fed the wheels of mills. This stretch was also lapped by a road paved with pebbles and paved by a "rizzarda" to allow the carriages to proceed easily. In 1659, by the will of Federico Borromeo, the construction of the Collegio Elvetico, today known as Palazzo del Senato, was started. It was intended for the education of the clergy working in Switzerland against the growing spread of Protestantism. The institution functioned with difficulty until the death of San Carlo. It was only with Federico Borromeo that things changed, and the Elvetico College (Bascapè, 1968) started to be more correct and profitable way that allowed the transformation of the entire building complex. The design and construction of this first courtyard, copied in the second one, gave the Collegio Elvetico (and the Seminario Maggiore which lived a parallel history to that of the Collegio) a particular architectural dignity, the cause of its subsequent political fortunes. But the works did not follow with the hoped speed and architects alternated, including Mangone, and later Richini. The elliptical solution of the facade to solve the connection between the facade of the college and the more advanced one of the church, and the attenuation of the discrepancy between the orientation of the facade itself -orthogonal to via San Primoand the oblique course of the Naviglio are his work. The fortunes and the tenants of the building alternated over the years and the work never ceased till the actual destination of the State Archives, still present. At the end of the 20s, beginning of the '30s, the first section of the inner circle was closed via Fatebene Fratelli and via San Damiano. At the beginning of the '30s, the inner circle was drained and then covered, leaving it dry. Because of the degradation of the covers, at the beginning of the '60s, all the inner circle is filled with sand and inert material.

The research method applied to research case studies: from primary and secondary data sources to digital models
The 3D survey of an artefact, a building or an environment is doubtless an efficient a powerful tool to study its shape, colour, position, relations with other objects or buildings, but it is not enough to learn all the information about them. The survey gives us the geometrical information about a subject, but to have a complete overview of its history we need to search and study monographies and bibliographies consulting the historical archives. This research followed this principle by collecting as much as possible textual and image information about the three case of study here presented. Especially the historical photos, paintings and drawings helped the authors figure out the original design and the outline of the Cerchia dei Navigli in the past. This was not trivial research, in fact the layout of the city changed significantly over the years, and it wasn't easy to reconduct the texts and the images found to a single historical period. The city of Milan passed through the bombings of the two world wars and through essential changes to the urban planning that led to the closing of most parts of the Cerchia dei Navigli. For these reasons, it was hard to identify landmarks in the images that help to reconstruct the places as they were in the past digitally. Nevertheless, the integration of primary data sources, i.e. the geometrical information about the object itself and the secondary data sources, namely the written and figure documents, permitted to digitally reconstruct each environment of the three case studies and bring them to life they were in the past.
The collection of the geometrical information considered using the latest technologies and methods in the survey field. Each case study planned a topographical campaign, a photogrammetric acquisition, and some terrestrial laser scanner stations.
The data elaboration took advantage of the multiple data sources to get the best from each instrument: • the dense point clouds coming from the laser campaign were collected, registered and then imported in a NURBS 3D modelling software to reconstruct the primitive geometries of the buildings and environments; • the photogrammetric approach both allowed to get high-resolution 3D textured models and orthophotos, which useful to map the NURBS models; • the topographic measurements were useful for checking the elaborations' correctness and merging the models from laser and photogrammetry into a common reference system.

Model generation and Information Mapping
The model generations stage involved applying scan-to-BIM requirements such as generation grades 9 and 10, oriented to advanced 3D mapping techniques (Banfi, 2020). In particular, NURBS allowed an appropriate transition from point clouds and mesh models to information models capable of maintaining high LOD, accuracy and LOI. Thanks to creating a large number of geometric elements corresponding to the constructive elements such as the bridges, the road surface, the filling tank, the gates, the mandrake and the spillway, it was possible to lay the foundations for 3D models able to correspond to the survey reality. The second key step to achieving a material connotation of each digitised element is an automatic synchronisation mode between modelling software and XR development applications. This synchronised mapping technique has allowed obtaining several benefits during the model creation process between MC Neel Rhinoceros, Autodesk Revit and Twinmotion / Unreal Engine: • use of a large number of orthophotos coming from photogrammetry; • mapping the models with images corresponding to the detected elements without using default textures; • views and edits the scene in real-time in both software applications (Modelling application and XR development platform); • reduces time-cost of XR development environments; • prepares 3D objects for visual scripting, moving from simple static models to Interactive Virtual Objects (IVOs); • provides support for all of the most popular VR headsets to take you from BIM to VR in seconds.

Information sharing and the development of visual programming languages: from informative models to interactive virtual objects
Thanks to the model generation stage have been possible to transform BIM, CAD, and NURBS model into a real-time rendering experience faster and more efficiently, enhancing the productivity, quality, and creativity of information during the reconstruction process. At the same time, the virtual reconstructions of three research case studies had to define a scientific approach able to tell and share different types of information to the final users such as professionals, students, and virtual tourists. In this context, the three XR projects have been developed with the primary purpose of entertaining and at the same time transmitting cultural and historical values of the old configurations, creating a common thread between present and past. At the centre of attention was the desire to create an effective and pleasant training experience for the virtual user by developing IVOs capable of responding to user input. It was crucial to apply and develop different blueprints to give life to the previously digitised objects. Unreal Engine introduces a visual scripting system and an interface that allows non-IT experts to program behaviours without directly acting on the code but using only a system of nodes. The nodes appear as panels of different colours, which users can connect to each other through arrows, allowing us to establish behaviours for our game. This allowed authors to build very advanced features without writing a single line of code. Figure 10 shows the comparison between the scan-to-BIM models and their virtual reconstruction according to specific historical phases. Furthermore, each scene has been enriched with the integration of IVOs such as information panels, multimedia files (video, audio, panoramas), descriptions and many other objects able to interact with the visitor and multi-devices (PC workstation, mobile phones, tablets, VR headset and laptop). Figure 10. The research method applied to the Conca dell'Incoronata: from scan-to-BIM objects to IVOs and the virtual historical reconstruction.

CONCLUSION
The method proposes a historical reconstruction of the main canals and water basins of the ancient Cerchia dei Navigli in Milan, intending to pass on its tangible and intangible values of the built heritage. The virtual reconstruction was supported by in-depth historical archives, historical photos, 3D surveys, HBIM. The latter made it possible to go beyond a simple 2D vector CAD representation. Thanks to the bidirectional information-object relationship, the authors could connect a large amount of information such as the characteristics of the materials (physical and mechanical), historical phases and multimedia files. On the other hand, virtual historical reconstruction has required new levels of interactivity and virtual experiences. Thanks to integrating different types of modelling techniques with the most open and used eXtended reality (XR) development platforms, has been possible to achieve new interactive forms of virtual-visual storytelling of the main water basins the Conca dell'Incoronata, Conca di Viarenna and Via Senato. The digital workflow has benefited from advanced post-production techniques of HBIM models such as 3D Mapping, real-time rendering, and the VPL to enhance the virtual experience through IVOs. Furthermore, thanks to specific 3D exchange formats, authors have transformed static mesh, point clouds and HBIM objects in XR objects to improve new levels of information sharing for different types of users (Architecture, engineering and Construction experts, virtual tourists, and students). Finally, particular attention has been paid to the latest generation tools such as the VR headset which allows to increase the level of immersion of the digital user, taking him back in time and allowing him to live in a historical context that can no longer be relived. Future developments will be oriented to enhance the level of interactivity and information of the proposed workflow. The main aim will be to integrate the other key points of the Cerchia dei Navigli, completing the VVS of one of the most important ancient European waterways.