COST EFFECTIVE SPHERICAL PHOTOGRAMMETRY: A NOVEL FRAMEWORK FOR THE SMART MANAGEMENT OF COMPLEX URBAN ENVIRONMENTS

: The development and urban planning of a modern city, nowadays, should be entrusted on the implementation of methods and techniques which require a management of complex information. The final goal is to support local authorities for the decision making. Finding data that are often heterogeneous but nevertheless connected to each other is useful to create a virtuous management model based on an empirical and objective study system. It will therefore be important to develop a system of data retrieval, analysis and management as accurate as possible, usable by all the actors involved in the governance of the territories. The article focuses on the implementation of an effective workflow for the management of complex urban data, the final goal of such framework, is the creation of a Smart City 3D Platform capable of providing innovative services for tax assessment and collection. In particular, it investigates over the potential of using spherical photogrammetry, to guarantee fast, low-cost and reliable acquisition time. The resulting 3D model has been then georeferenced with GNSS coordinates to ensure the desired precision, while the assessment of the model has been done using laser scanner data as a ground truth. The point cloud obtained from the processing can be managed and edited in a WEBGIS, which merges 2D (cadastral register) and 3D (point cloud) data. The project is the result of the collaboration between the Università Politecnica delle Marche and the Company Andreani Tributi srl, with the aim of collecting information about the advertising structures present in the city of Brescia (Italy) for tax assessment.


INTRODUCTION
Today's urban environments are increasingly complex, making their monitoring expensive and time-consuming. This condition represents a bottleneck for local administrations and specialised operators, who deal with a complex system in terms of both management and planning. These issues are driving stakeholders to develop and deploy new strategies for intelligent management. In this light, emerged the need to remodel urban areas and rethink the services they offer, ensuring greater efficiency and liveability (Sallis et al, 2002). To this end, the monitoring of urban furniture plays a pivotal role as regard its state of wear, the assessment of the state of conservation of the historical and cultural heritage, the control of road signs (to guarantee users comfort and safety on the move), the monitoring of the correct functionality of public lighting devices, as well as controlling advertising systems, ensuring their compliance with current regulations. Smart Cities focuses on new ways of urban governance, in which local governors have knowledge management based on an organisational strategy (Ratcliff, 1974). The planning system includes preparation, evaluation, implementation and control phases. The intention of many local authorities is to implement the Smart City concept, which includes a set of urban planning strategies aimed at optimizing and innovating public services (Kourtit et al., 2016). Today, citizens live surrounded by a great quantity of data, generated both by people and sensors, which transform the way it is done business, the way government manages public resources, the way research and science look to * Corresponding author the future. A key point of Smart Cities is the digitization and classification of all this data (Nuaimi et al., 2015), an approach involving the use of new technologies, possibly at affordable costs. Smart Cities bring a new concept and model, which applies the new generation of information technologies, such as Internet of Things, cloud computing, big data, and space-geographical information integration, to facilitate the planning, construction, management, and smart services of cities (Toh et al., 2020). Nowadays, decision-making strategy is focused towards the exploitation of these innovations. By collecting and updating geo-referenced data to support urban and non-urban decisionmaking strategies, might represent the turnkey for undertaking such ambitious way. In light of the above mentioned considerations, the objective of this work is to present an effective workflow developed for the intelligent management of complex spatial data. Toward this end, the paper describes the creation of a Smart City framework, designed to facilitate the management of 2D/3D data; the final goal is to replace the manual operation carried out by the expert operators, who will be able to perform the task remotely after the data acquisition process. It is, in fac,t a common practice by local authorities, to perform direct surveys, with time consuming campaigns. Our idea was to substitute manual operations with fast and low cost data acquisition, exploiting 360° panoramic images. Indeed, by exploiting Spherical Photogrammetry (SP), it is possible to create the 3D model of the city, represented through point clouds. Despite SP proved to be a valuable technology for Cultural Heritage (Fangi, 2019), few works focus on its potential for Smart Cities. This technique provides a detailed visual representation of the surrounding environment, besides an objective and metric survey. The project was driven by a joint venture between the academic world and a private Company. Specifically, the desired result was the creation of a management service to facilitate the detection of illegal advertising panels, verified through the integration between the 3D model and the cadastrial data. This integration of geo-referenced data into a cadastral register will lead to a whole series of innovative services aimed at ensuring transparency and traceability of tax collection and the fight against tax evasion. The workflow can thus summarized as follows: after the collection of images through 360° cameras, the datased is then processed getting a complete 3D reconstruction of the city. The 3D model has been validated with laser scanner data and georeferenced with GNSS positioning. The computation yields good accuracy, in line with the tollerance allowed. Finally, a WEBGIS has been developed, which exploits WebGL interface to interact with the 3D model and a GeoDB to manage cadastrial and fee information in an all in one solution. Below is shown the synthetized workflow ( Figure 1).

RELATED WORK
The world population has increased significantly in recent decades, as its longevity. It is expected that about 70% of citizens will live in urban areas by 2050. This quick increase will cause unavoidable displacement of people from rural areas to more urbanized areas 1 . The aggregation of new citizens in a disorderly way, accompanied by new needs of residents and different standards of living, represent new challenges for governments. They will have to deal with issues related to waste disposal, scarcity of financial resources, increasing air pollution and associated risks to human health, increased traffic and maintenance of infrastructure to avoid degradation and wear (Joshi et al., 2016). Local administrators will be expected to make ever right decisions in support of the welfare of citizens. To support these choices, it is appropriate to think of a new city model, the so-called Smart City: an infrastructure that allows the use of Information and Communication Technologies (ICT) such as Internet of Things (IoT), Big Data, Cloud Computing "Mobile broadband" and "Short range wireless" (Borda et al., 2017) applied to urban management. A Smart City is a city that employs an intelligent strategy of systems oriented to the management of complex information, mainly collected in real time (Pedro Filho et al., 2018). In addition, its development requires a common platform that provides the visualization of heterogeneous information and sensor network sources. A key role is played by Geographic Information Systems (GIS), which have an effective application for the connection and interoperability of data systems. It also provides an excellent basis for: efficient urban and spatial planning, spatial analysis, expedited decision making, implementation and verification of the maintenance and monitoring plan. (Xiaoming et al., 2015). In this context, visualisation technologies are varied, going from simple to advanced. The data that can be included are high resolution photogrammetric images, cadastral data, urban planning data, municipal area photos, building addresses, 3D data, service infrastructure and all kinds of corporate data related to the work of municipal services. Therefore, the management and interface of the static and dynamic data in 3D fashion environment goes under the umbrella of CityGML (Chaturvedi et. al, 2020) which exploits open systems and international standards such as those provided by Open Geospatial Consortium (https://www.ogc.org/). An excellent way to obtain the data is given by the outdoor panoramic photography, applied not only for commercial purposes, but also for the three-dimensionality of the metric documentation especially in the fields of architecture and archaeology (Fangi, G. 2012). The advantages in the use of SP are the high resolution of the images obtained, the FOV up to 360°, the low cost, the completeness of the information and the speed of acquisition. The goal of SP is to allow anyone to make documentation in any location, both indoor and outdoor, with simple and cheap devices. Therefore, currently, the most common practice for creating low cost cylindrical or spherical panoramic is generally relied on the collection of linear arrays and rotating camera, with a very high metric performance, on a panoramic head. (Luhmann e Tecklenburg 2004). In addition, in order to provide 360° coverage with accurate 3D measurement capabilities, using panoramic cameras, strict sensor and system calibration procedures have been implemented to reconstruct a highly detailed and complete 3D urban model of the road environment. (Blaser ad al., 2018). In this regard, specific research has recently been carried out. In the field of Heritage Cultural, the development of a 3D-GIS of the urban environment and the possibility of interacting with a huge amount of semantic information contained in the 3D geospatial model of the city (Gonçalves et al., 2016.). These platforms are able to view, store, analyze and share 3D data to improve decision making, planning or problem solving. This methodology of work can also be applied to the management of infrastructure networks, to perform graphical analysis and interactive data visualizations, to simulate real working environments, even remotely. (Al-Rawabdeh, et al., 2014). The boost to provide data to all users and to create cities suitable to the needs of citizens, led the European Community to launch the i-SCOPE project, "Interoperable Smart City services through an Open Platform for urban Ecosystems". Its aim is to develop an open platform based on a 3D model of the urban territory (3D CityGML Model) able to provide innovative and intelligent services for Smart Cities through the precise modelling of urban components, an integrated and interconnected set of information flows related to the urban context. (Buffoni et al., 2013). A similar case was performed in the province of Trento, with the integration of the 3D platform i-Scope to the cadastre (Prandi et al., 2014). In doing so, various types of data have been integrated, ensuring the interoperability between taxes and information to each three-dimensional object represented, from different sources. The work presented in (Kwiatek et al., 2014) is the one closest to the research here presented. It explores the potential of immersive videography in photogrammetry from multiple cameras, with requirement and specifications, allowing the user to take measurements on advertising panels from panoramic images. In (Liu H., et al. 2019) a framework for illegal billboard advertising detection is proposed, based on machine learning techniques.

CASE OF STUDY
As pilot case to demonstrate the efficiency of the proposed workflow, it is tackled the issue of advertising panels. The illegal appropriation of such advertising posters is increasingly pressing local authorities, because of both the lack of tax collection and of urban decorum. Their management and monitoring are either handled by the municipalities themselves or contracted out to external collection agencies. The Company financing the research project, Andreani Tributi srl, takes care the collection of local tax on advertising in the City of Brescia (Italy). The traits chosen in this analysis are part of an area that has already been manually detected by the Company's operators. This choice is due to study the benefits and efficiency of working remotely. The two roads section, Via San Polo and Via Sant'Eufemia, have both billboards and shop signs. In Figure 2 is showed the road in Via San Polo, with the trajectory carry out by user and some image extracted from immersive video.

ACQUISITION AND METHODOLOGY
In this section the methodology of data collection is described. More specifically, the SP acquisition and processing is compared with ground truth data (from laser scanning point cloud), in order to validate the methodology. A brief description of the existing data to be integrated in the framework is provided as well.

Data acquisition
The experiment was conducted using the Nikon Key Mission 360° camera ( Figure 3a) and extracting the spherical images from video. The videos were performed in different ways according to the road sections and their relative metric length: in the first case, in Via San Polo, with an operator walking along the road; in the second case, in which the operator, placing the camera at the end of an extensible pole, shoot a video proceeding on a vehicle along the route of interest. The use of video shooting results in a redundant high frame image and a loss of accuracy of the photogrammetric orientation. This leads to the search of a methodology that optimizes acquisition and return with the lowest projection error. It was decided to use a GNSS receiver to survey, on the ground, coordinates of clearly visible points such as the edges of machicolations on the road surface or special signs of road markings. The targets placed on the ground have been acquired by the GPS HiPer HR 2 (Figure 3b) with RTK (Real Time Kinematic) method. On top of GNSS satellite signals an RTK receiver takes in an RTCM correction stream and then calculates your location with 1 cm accuracy in real time. The rate varies between receivers (Base and Rover) but most will output a solution at least once per second. This step allows to obtain a 3D point cloud with high precision. This manual procedure was very time-consuming due to the continuous stop and go. Every 300 meters, the operator not only surveyed the point, but also took a picture of the surrounding environment in order to avoid errors and doubts in the positioning of the marker during the processing phase. This happened because the panoramic camera is not inside equipped with GNSS receiver. On top of GNSS satellite signals an RTK receiver takes in an RTCM correction stream and then calculates its location with 1 cm accuracy in real time. The rate varies between receivers (Base and Rover) but most of them will output a solution at least once per second. Parallel to the video shooting, a survey was carried out with the Kaarta Stencil 2 laser scanner ( Figure 3c) on the same road sections in the same way as described above. Its contribution has been chosen to obtain a point cloud model as ground truth. Indeed, in this paper has been described the method for evaluating the point clouds obtained from panoramic images and the resulting differences with reference model from Kaarta Stencil 2 as described in paragraph 5.1.

Data processing
The dense point cloud was processed using a software based on Structure from Motion algorithms by inserting spherical images. The software allows the combination of close-range images with the identification of homologous points supporting the automatic calibration of the camera, allowing the internal and external orientation. The first operation in data processing was the extraction of equirectangular frames from panoramic video. At this point it should be considered the number of frames to use for the image alignment. In the literature it is recommended to choose a minimum number of frames that is a function of the video capture rate (Kwiatek et al 2014). In addition, it should be considered also the speed of the vehicle that in the case study was 50 km/h which is converted in 13.89 m/s. This means that during every second of driving at least 14 panoramic photos should be taken (Figure 4). Even though the baseline value is very short and in theory guarantees the overlap with a smaller number of images than calculated, this was not possible. Due to the low image resolution and the incident light during the survey, it was necessary to work with an equal or even higher number of frames than the above calculation in order to correctly orient the cameras with the rest of the model. This caused major slowdowns with the alignment process. The manual frame extraction operation is a fundamental and at the same time complex phase, because the user have to divide the video into several ranges, according to the speed of the vehicle. At the end of this processing above mentioned, the frames have been imported and placing the Ground Control Points. These markers, manually inserted by the operator in as many images as possible, have the task: to guarantee an absolute orientation of the cameras, ensure a high number of homologous points between the different panoramas and georeferencing the model. Finally, the dense clouds of the sections under investigation have been processed. Several tests were carried out to try to reduce and minimize significantly the alignment and processing times: since this is a phase in which the software performs photogrammetric processes, it was decided to apply masks mainly to the sky and the silhouette of the vehicle in all the images processed. Below a view of dense clouds in Figures 5, 6 and 7, instead the Tables 1,2 contain the data obtained from processing.  . Some sample of advertising panels visible in the point cloud

Kaarta Stencil 2
Kaarta Stencil 2 ( Figure 1c) is a stand-alone, light weight SLAM instrument, with an integrated system of mapping and real-time position estimation. In addition, it is a hand-held of limited size, which allows quick and easy 3D mapping by hand made. This choice was made to survey the environment around the road. As for the photogrammetric survey, the road was run both on foot and by car. The laser scanner has been assembled on a small pole held by hand. KAARTA Stencil 2 depends on LiDAR and IMU data for localization. The system uses Velodyne VLP-16 connected to a low-cost MEMS IMU and a processing computer for real-time mapping. VLP-16 has a 360° field of view with a 30° azimuthal opening with a band of 16 scan lines. The data acquisitions were captured using the KAARTA Stencil 2 default configuration parameters, set in order to use the instrument in structured outdoor environments. Specifically, these settings include default values (Table 3)  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2020, 2020 XXIV ISPRS Congress (2020 edition)

Existing information
As stated in the introduction section, the management of urban information is made manually, with data acquisition campaigns made by expert operators. This task is performed using an ESRI ArcPad mobile application. In the office the basic data package is prepared by ArcMap and loaded on the pads before going out for the survey. The supporting data used within the application are cadastral shapefile and urban mapping which in turn are related to the particle owners. The exported data are in general with WGS-84 reference system. The data are stored in a Personal Geo-Database (Microsoft Access file). On returning to the office, the survey data are downloaded, Excel files are exported by type of tax and stored in a platform. The data acquired during the survey are: -type of tax to be ascertained for each cadastral parcel (ICP posters/advertising, TOSAP driveways, PGIP implant plan) referred to each point registered; -Photographic image referred to the point collected, as mentioned above.
In Figure 8 is represented the interface of the current software of the firm, during the survey in the field.

Comparison of the results
At the end of the acquisition phase with KAARTA Stencil 2 the information about the configuration setting, the estimated trajectory (Table 4) and the 3D point clouds (Figures 9, 10), are stored in a folder created automatically by the mobile laser scanner processer every operation of survey. If the elaboration of 3D point cloud is made by mobile laser scanner, the post-processing phase deserve some explanations.
Using the freeware version of CloudCompare 3 (Open Source Project), it is possible to evaluate the remoteness of the model in comparison with a reference model. In this program, it is compared the data taken from spherical camera and Lidar. The 3D data, saved in .ply (Polygon File Format) format, can be opened in CloudCompare that allows these main functions: registration and alignment of point cloud, manual or automatic cleaning and segmentation of area study. By adopting the verticality of the scalar field, the software calculates the normal along the z axis. This operation ensures the elimination of curved objects or noise, returning the vertical surfaces flat and well defined. The Kaarta Stencil 2 is not integrated with a GNSS system, which would otherwise ensure a georeferencing of the output database. Therefore the user must manually position at least three targets to ensure its correct positioning in space. The unique method is to select control points on the horizontal road, because its reflectance makes it clearly visible and recognizable to the human eye. So a first solution was to use the Control Points taken by the GNSS survey. In the end, the cloud was segmented by the user with a 2D polygon on the object, in order to obtain a 3D model closest than the outcome obtained by photogrammetry. With this procedure only the points inside the polygon are kept. After that, the point clouds are imported into CloudCompare. to evaluate the distance between them, using the C2C tool. In order to make the comparison, it has been set the point cloud by Kaarta Stencil 2 as the reference cloud to be compared with the cloud obtained by photogrammetry. To validate the result, at the end of the processing, a Gaussian curve has been exported, with a max The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B4-2020, 2020XXIV ISPRS Congress (2020 distance of 0.50 m. This tolerance allows the operators by the Andreani Company, to calculate the fee on the advertising panels according to the approximate area for excess of half a meter. The computation of the differences confirmed the metrical accuracy of the models by immersive video. The Figure 14 and Figure 15 shows some significant results deriving from this procedure. The pictures show that the greater differences correspond to the curve surfaces and to the areas where the reference point cloud had no information. In Table 5 are reported the statistical parameters derived from the analysis with C2C tool, such as maximal distance, average distance and standard deviation.  Table 5. Statistical results from the distance comparison between photogrammetry point cloud and the Kaarta Stencil 2 point cloud

WEB tool for the visualization of point clouds
More and more large digital data are displayed and managed in real time, thanks to platforms and database, which are based on the latest software. Many of these systems not only query the animation sequences of 3D models, but also allow further analysis and investigation when connected to databases. Many visualization tools in the field of cultural heritage have been created in recent years, allowing the development of new ways for the preservation and promotion of physical cultural assets (Fangi et Figure 16, in which the web renderer provide a toolbox that allow the user to obtain metric information about the model and the surrounding environment in case of outdoor scenario. Besides, Potree allows a hyper-link as image, as showed in Figure 17. In this way is possible to control with more detail the objects that are not clear in the point cloud model.  Figure 17. Display of georeferenced spherical image using Photo Sphere Javascript library

Data management framework
Albeit the results of the previous steps are effective, the lack of interoperability about data processed and existing repositories hamper their real exploitation. To overcome such limitation, an integration among the system has been developed. The WEBGIS interface offers various spatial and topographic interaction tools, such as distance, area, volume, angles and coordinates. In this platform it is possible to explore the point cloud thanks to the pan, tilt and zoom function. A useful feature is the annotation command that allows the selection of different levels of point cloud data. Finally, the link command is provided to access the GIS platform to support the user to query additional data. The file is geo-referenced and connected to a web server such as PostgreSQL/PostGIS, the data collected in this way is aggregated and periodically exported, reprocessed and make available through the open source Web-GIS platform as Lizmap. The architecture of data management is depicted in Figure 18.

CONCLUSION
This work stands as a guide for the management of public assets specifically for tax collection; it is a model that integrates different systems such as SP, information technology and data communication to meet the needs of local authorities. Moreover, these features are the basis for the creation and operation of the Smart City. It can be highlighted how the development of this new methodology can bring benefits and have a positive impact: in terms of time, costs for the public finance, for the speed of data acquisition on the territories examined and especially for the small number of human resources employed in the work. The framework described in this article are promising. The Nikon Mission Key 360 has proved to be suitable for metric reconstruction even though it is an inexpensive tool; the precision values and the distance between clouds described in paragraph 5.1 can be compared with a three-dimensional model obtained with the Mobile mapping system technology. There isn't any doubt that this system is a good way to acquire large amounts of data without losing accuracy. The visualization of large point cloud data in web viewers such as Potree, completely replaces the operations that were performed on site, as described in paragraph 5.2. The goal is to create an atlas in the tax management of the cities in which the Company gets the assignment.
Regardless of the quality of the point cloud, this research opens the way to the use of fast, easy and universal tools because even a not highly qualified staff can easily and rigorously survey entire neighborhoods and urban environments. The aim is to document municipal areas and query three-dimensional models to answer at citizens' needs.