GEOSPATIAL DATA PROCESSING FOR 3D CITY MODEL GENERATION, MANAGEMENT AND VISUALIZATION

: Recent developments of 3D technologies and tools have increased availability and relevance of 3D data (from 3D points to complete city models) in the geospatial and geo-information domains. Nevertheless, the potential of 3D data is still underexploited and mainly confined to visualization purposes. Therefore, the major challenge today is to create automatic procedures that make best use of available technologies and data for the benefits and needs of public administrations (PA) and national mapping agencies (NMA) involved in “smart city” applications. The paper aims to demonstrate a step forward in this process by presenting the results of the SENECA project (Smart and SustaiNablE City from Above – http://seneca.fbk.eu). State-of-the-art processing solutions are investigated in order to (i) efficiently exploit the photogrammetric workflow (aerial triangulation and dense image matching), (ii) derive topologically and geometrically accurate 3D geo-objects (i.e. building models) at various levels of detail and (iii) link geometries with non-spatial information within a 3D geo-database management system accessible via web-based client. The developed methodology is tested on two case studies, i.e. the cities of Trento (Italy) and Graz (Austria). Both spatial (i.e. nadir and oblique imagery) and non-spatial (i.e. cadastral information and building energy consumptions) data are collected and used as input for the project workflow, starting from 3D geometry capture and modelling in urban scenarios to geometry enrichment and management within a dedicated webGIS platform


INTRODUCTION 1.1 Motivation
A rich understanding of urban areas is nowadays paramount and increasingly required by the fast urbanization of the world and a necessary continuous monitoring.According to the UN World Urbanization Trend (http://esa.un.org/), the world is undergoing the largest wave of urban growth in its history and forecast future scenarios confirm this trend: in 2050 about 66% of the world population will live in urban areas.The need for a more efficient mapping, understanding and management of the urban ecosystem is thus evident, especially since a development towards a "smart city" scenario is envisaged.Within this context, 3D geospatial data and geoinformatics technologies are expected to play an essential role in supporting smart city strategies.In particular, urban 3D reconstruction has gained a significant attention in recent years and several potential application fields have emerged, e.g.energy assessment, space and traffic management, 3D cadastre and cartography, noise and illumination pollution simulations and analyses, emergency management and virtual reality, etc. (Biljecki et al., 2015).These various domains urgently demand the development of advanced methods for efficient, automated creation, updating and maintenance of 3D city models.Indeed, despite the efforts done by different scientific communities (photogrammetry, remote sensing, geo-informatics, computer vision and computer graphics), there are still many unsolved problems.The potential of 3D building models is mainly confined to visualization purposes (e.g.Google Earth), although they can provide more than just a 3D visualization of real world.Furthermore, the reconstruction of complex urban scenarios is still challenging: significant manual editing or unrealistic assumptions are usually required when it comes, for instance, to the modelling of historic city centres (Toschi et al. 2017).Finally, with regards to the data type used as input to the modelling process, algorithms based on point clouds generated by airborne photogrammetry do not usually exploit the full potential of most recent multicamera systems (Remondino et al., 2016).Although oblique photogrammetry is rapidly maturing, the use of slanted views for façade reconstruction is still limited by image resolution and occlusion issues (Haala and Rothermel, 2015).Therefore, although data and technologies are mainly available, the major challenge today is to create as much as possible automatic procedures that make best use of them for the benefits and needs of public administrations (PA) and national mapping agencies (NMA).

The SENECA project
The integration of expertise in the geospatial data modelling and geoinformatics technologies may bring 3D city modelling a step forward, i.e. towards a real use, exploitation and valorisation of urban 3D reconstructions.With this in mind, the SENECA project (Smart and SustaiNablE City from Abovehttp://seneca.fbk.eu) was started in 2015 within a collaboration framework of among research partners, public administrations and SME players.The project (Nocerino et al., 2017) seeks to answer the growing demand for (i) high quality spatial data, (ii) efficient methods for sharing and updating geo-information and (iii) a sustainable management of natural resources and energy.More specifically, the project aims to investigate state-of-the-art processing solutions in order to (i) efficiently exploit the aerial photogrammetric workflow (aerial triangulation and dense image matching), (ii) derive topologically and geometrically accurate 3D geo-objects (i.e.building models) stored in a 3D webGIS and (iii) link geometries with non-spatial information within a scalable, editable and expandable 3D web-based environment.The latter is designed to provide potential endusers (e.g.public administrations, real estate companies, security companies and general public) with a useful tool for building documentation and management, urban planning, energy audit and PV-potential estimation.Therefore, in order to support and tailor the project to the actual needs of the urban community, PA, private organizations and general public are actively involved.Thanks to this collaboration, heterogeneous pieces of information -ranging from spatial and cadastral data to energy requests and consumptions -are collected, processed, merged and shared.

Paper objectives
The paper aims to present the results of the SENECA project, from 3D geometry generation in urban scenarios to geometry enrichment and management within a dedicated webGIS platform.Both spatial and non-spatial data are acquired and used as input to the developed methodologies (Section 2).The latter are tested on two case studies, i.e. the cities of Trento (Italy) and Graz (Austria).Different airborne camera systems are adopted to capture imagery over the two cities and different non-spatial data are collected for the two case studies.This leads to distinct workflows adopted (Section 3) and outcomes delivered (Section 4).Although the project is not yet completed for the time being, critical remarks emerged from the experience are discussed (Section 5), together with further improvements, extension and value-adding processes envisaged for the platform in the next future.With respect to other similar publications (Gál et al., 2009;Agugiaro, 2014;Nouvel et al., 2015) in this article we deal with the entire pipeline, from 3D geometry generation, to enrichment and web publication.

Test areas
The workflow developed within the SENECA project is adopted on two test sites that pose significant reconstruction challenges.The first project area is the city centre of Trento, located in the Italian Alps, with an approximate size of 3.5 km by 1.5 km.The city, with a population of about 117,000 inhabitants, lies on the banks of the river Adige, in the homonymous valley.It presents old buildings with complex and varying shapes (Romanesque, Medieval and Renaissance styles), located in a densely built city centre (Figure 1, top).The outskirts contain mainly residential and commercial buildings, with single and terraced house.The second project area is located in the city of Graz (Austria), and covers an area of ca.3.0 km by x 1.5 km.The city (280,000 inhabitants) lies in a basin on the banks of the river Mur, southeast of the Alps.Its oldest centre (Figure 1, bottom) presents a medieval main square surrounded by narrow streets, with Renaissance and Baroque architectures.The immediate outskirts, more recent and less densely built-up, mainly feature single houses in a residential area.

Spatial data
High resolution aerial imagery represents the main data source for the modelling process of the project.Compared to airborne LiDAR and traditional 3D reconstruction algorithms based on the generated point clouds (Haala and Brenner, 1997;Kada and McKinley, 2009), optical images normally feature higher spatial resolution and thus the potential for extracting 3D point clouds at a remarkable resolution, accuracy and reliability.Although image-based methods suffer from well-known issues like shadows, occlusions and poor contrast (Brenner, 2005), recent developments of available technology and methods encourage the use of images and derived 3D points for urban digital reconstructions.Two different camera systems are adopted for the photogrammetric surveys.The Trento dataset consists of 397 images captured with the large-format nadir-looking digital camera UltraCamXp.The image block is flown with a traditional 80/60 overlap (along/across track) and the flight plan is designed for an average GSD of 10 cm.The Graz dataset comprises 180 images and is acquired using an UltraCam Osprey I multi-camera system (one large frame nadir camera with six oblique looking cameras along the four cardinal directions).The area is flown with an overlap of 75/65 (along/across track) and a nadir GSD of about 12 cm.Both datasets include ground truth information in form of GPSsurveyed ground control points (GCPs) and check points (CPs) acquired with a mean accuracy of 8 cm (Trento) and 5 cm (Graz).Furthermore, the exterior orientation parameters are available as GNSS-IMU observations.More details about the camera systems and the datasets are listed in Table 1.Additionally, a thermal flight is performed over the city of Graz adopting a TABI-1800 thermal sensor.The delivered orthophoto features a spatial resolution of 60 cm and is processed using a LiDAR DSM available for the area.Finally, land cadastre maps in vector format covering the whole city of Trento are provided as shapefiles (nominal scale of 1:2,000).The latter contain, among the rest, the building footprints, each linked with a unique land parcel ID.

Non-spatial data
Several pieces of non-spatial information are provided by public authorities and private organizations for the city of Trento.These include:  energy performance certificates of few buildings, including both residential houses and public utilities.They mainly list energy consumptions, carbon dioxide emissions and energy efficiency indexes. sources of artificial night-time light (single spots), like streetlights, including spatial distribution and light emission information. data from the register of buildings, including information about each property unit (e.g.owner, number of floors, number of rooms, building surfaces, property category, etc.).

3D geometry capture
Given the necessity to cover large areas, airborne data represents the main input for 3D geometry capture in urban scenarios.Several 3D modelling methods have been recently developed in order to reconstruct the geometry of urban environments using aerial imagery (Kluckner and Bischof, 2010;Nex and Remondino, 2012;Bulatov et al., 2014).Nowadays the base information is derived from dense point clouds extracted with state-of-the-art multi-stereo matching algorithms (Haala, 2013;Remondino et al., 2014).When traditional nadir-looking cameras are adopted, the workflow is mainly straightforward: starting from the aerial triangulation (AT) results, many commercial matching tools can generate DSM raster and cloud representations at a GSD-level resolution.When it comes to the processing of oblique imagery for 3D information extraction, the task becomes not-trivial.
Both image orientation and dense matching are aggravated by several challenges, e.g.grater illumination changes, multiple occlusions, large scale variations and high sensitivity to noise (Wiedemann and More, 2012;Rupnik et al., 2013;Gerke et al., 2016).Furthermore, from a geometric point of view, the traditional 2.5D processing for DSM production from nadir images should be replaced by a more compelling modelling in "real" 3D space (Haala and Rothermel, 2015).This inevitably requires the development of ad-hoc filtering (and meshing) algorithms for the derived 3D points.
These two approaches are both employed within the SENECA project (Figure 2), using state-of-the-art software solutions, either commercial or open-source, i.e.Pix4D (www.pix4d.com/),SURE (www.nframes.com)and MicMac (http://logiciels.ign.fr).The nadir imagery collected over Trento is processed using the traditional pipeline, consisting in image triangulation (with self-calibration) followed by dense image matching and DSM generation.The output is a 2.5 point cloud resampled with a uniform spatial point distribution in the XYplane.On the other side, the Graz oblique dataset is first oriented by testing several strategies (Rupnik et al., 2015).Then 3D multi-view matching and filtering steps are applied (Rupnik et al., 2014).In this case, an unstructured point cloud with spatially heterogeneous point distribution is generated.

3D geometry modelling
Few years ago, photogrammetry was coupled to manual or semi-automated methods in order to generate 3D building models (Gruen and Wang, 1998).Automation started with the first dense LiDAR-based point clouds (Suveg and Vosselman, 2002).Nowadays, it is common to produce 3D city models (large polygonal shapes or single 3D buildings) from 2.5 or 3D point-based representations (Lafarge and Mallet, 2012).A geometrical simplification is then an essential step to reduce data complexity and allow for an efficient management and real-time rendering of the final output in web-based platforms.Several 3D reconstruction methods have been developed (Haala and Kada, 2010;Musialski et al., 2013).From an algorithmic point of view and focusing on building modelling, methods can be traced to two main categories:  reconstruction through template-fitting: building models "from scratch" in order to best fit the given point clouds (Kada, 2009;Sampath and Shan, 2010).Generally, these methods first apply an interactive pre-segmentation of the points, then try to detect buildings by searching for selected types of roof shapes (e.g.flat, gable, hip roofs, etc.).Building footprints, when available, can be adopted to support the process, after a non-trivial step of splitting and/or merging.The output of these model-based methods can feature different Level of Details (LODs), according to the OCG standard CityGML (Gröger and Plümer, 2012).Generally, they provide distinct roof structures, but are restricted to planar vertical facades (LOD2-compliant building models).These approaches, although providing impressive and convincing 3D models, lack from generality and build upon strong building priors on symmetry and roof typology.
 reconstruction through DSM or 3D mesh simplification: this class of algorithms leans on the concept that buildings are "contained" in a detailed 3D mesh or 2.5D DSM and seeks to simplify the meshed raw data until it meets ad-hoc geometric and semantic criteria.Different mesh simplification approaches have been introduced, e.g. based on dual contouring (Zhou and Neumann, 2010).These datadriven reconstruction methods are able to model roof shapes of arbitrary complexity and to extract building models in accordance with the desired abstraction level.Although more close to the raw data and more general, however, these approaches may lose semantic information and are often designed to deal with a specific type of input data.Both aforementioned approaches are employed within the SENECA project (Figure 3).The DSM cloud extracted from nadir imagery over Trento is used as input for the generation of LOD2-compliant building models.This is carried out by fitting roofs primitives on the presegmented data using the tridicon/Hexagon suite of tools (www.tridicon.de).Furthermore, LOD1 polyhedral models are reconstructed by extruding available cadastral footprints.Finally, ARCHCAD (www.graphisoft.com)supported by data collected from the register of buildings, are adopted to generate architectural 3D models (LOD3-compliant) of few buildings of interest.
For the Graz case study, a detailed 3D mesh is firstly generated from the filtered 3D point cloud, in order to keep details visible on oblique imagery and reconstructed on facades.Then, the mesh is simplified by reducing its number of faces, in order to allow an efficient uploading and rendering.

3D geometry enrichment and management
The extension of traditional database management systems in order to handle huge amount of spatial data has seen increasing amounts of research work in recent years (Gál et al., 2009;Lewis et al., 2012;Agugiaro, 2014;di Staso et al., 2015).
Focusing on 3D city modelling (Biljecki et al., 2015), the reconstructed geometry can be enriched with many pieces of non-spatial information, allowing for both visualization-based (e.g.visibility analysis, urban planning, 3D cadastre, emergency response, etc.) and non-visualization (i.e., when the visualization of 3D models is not required, being the spatial operations, such as estimation of the solar irradiation, estimation of energy demand, etc., stored in databases) usages.This relies on the creation of a scalable system designed to store, manipulate, analyse, manage and visualize different types of spatial and non-spatial data, and the connections among them.
Within the SENECA project, a service platform accessible from a web-based client was developed in order to manage both "data containers" (i.e.building models) as well as associated information (i.e.non-spatial data collected within the project activities).The architecture of the platform is shown in Figure 4 and features a multi-layer structure.Geospatial data are first adjusted within the "Raw data management layer", developed in NodeJs (www.nodejs.org), in order to make them compatible with the platform and allow for an efficient management of information along the entire chain.
Both spatial and non-spatial data are then stored within the platform database (DB), i.e.PostresSQL/PostGIS, accessible through the "Data access layer".Geometries reconstructed at different levels of details populate the DB and are associated with their corresponding non-spatial pieces of information via unique identifiers.Users permissions and filters are managed through the "Business logic layer", developed in NodeJs.It handles different types of users and, correspondingly, different types of permissions associated with them: this allows to handle data visibility and manipulation, and limit accessibility to sensitive and confidential information.The "Service layer" deals with data exchange with other layers through a RESTful architecture.Additionally, it provides public API for a limited data exposure and data import, in order to allow for a potential platform enrichment via integration with third-party software.Finally, the "Presentation layer" provides the web application that makes the data available to the end-users.It is developed in AngularJS (www.angularjs.org),an open source framework with a Model-View-View-Model (MVVM) architecture.In order to exploit the potential of 3D data, increase the user experience and the data representation quality, an in-house customized version of the open-source NASA World Wind (www.webworldwind.org) was adopted.It allows for geometry viewing, navigation, texture projection and map layers rendering.

Photogrammetric 3D reconstruction
AT is performed with GCPs and GNSS-observations of camera centres, included in the bundle block adjustment process as observed unknowns.For both datasets, the AT accuracy reaches values below the mean GSD of original imagery.In Table 2, the root mean square errors (RMSEs) on CPs are reported for both case studies as measures of adjustment accuracy in object space.
It is clear the benefit in Z from the addition of oblique images (Graz), mainly due to the improved intersection geometry of the rays (Rupnik et al., 2015;Moe et al., 2016).
Results of the dense image matching step are shown in Figure 5.The 2.5D DSM cloud extracted from the nadir-only dataset (Figure 5, top) features more than 5.5 billion points corresponding to a XY-grid size of 10 cm.As expected, reconstruction of the building facades is strongly limited by visibility issues and may be compensated for by the integration of data collected from terrestrial viewpoints.On the other hand, the completeness and density (1 GSD) of the 3D point cloud extracted from the Graz oblique dataset (Figure 5   On the other hand, a subset of the 3D mesh generated from the 3D point cloud of Graz is shown in Figure 6 (bottom).In this case, the detailed geometry can be visually enriched by either the RGB texture transferred from the VIS-orthophoto (left), or the thermal one projected from the TIR-orthophoto (right).

3D geo-database system
The present main platform potentialities include 3D navigation, data visualization and rendering (incl.sources of artificial nighttime light), data query and editing.Specifically, the user can navigate through the urban environment at different altitudes and camera angles.Figure 7 shows the platform web-interface.
The base map is provided by OpenStreetMap, over which building models are visualized.Depending on the altitude (i.e., the zoom level), the user can interact with (i) LOD1-compliant models at urban scale (Figure 7, left), (ii) LOD2-compliant models at neighborhood scale (Figure 7, middle) and, finally, (iii) LOD3-compliant models at single building scale (Figure 7, right).This allows for a real-time visualization and rendering, enhanced by texture projection, available at the second and third levels of detail (i.e. at neighborhood and building scales).Furthermore, it is possible to query the spatial data in order to retrieve the corresponding non-spatial information (section 2.3) and vice versa.This can be carried out either over the whole building population, or over a subset of it segmented through filtering operations.Non-spatial data (e.g.cadastral information, energy performance indexes, energy consumptions, etc.) is showed in pop-up windows, where information can be edited, enriched and updated.Finally, using energy parameters provided in building certificates and building geometry, an energy map (heating loss) can be computed and visualized as additional building texture (Figure 7, right).

CONCLUSIONS AND FUTURE WORKS
Results achieved within the framework of the SENECA project show that the integration of expertise in the geospatial data modelling and geoinformatics technologies development domains may lead 3D city modelling a step forward, i.e. towards the actual exploitation and valorisation of urban 3D reconstruction.The project adopted both traditional data and techniques (e.g.nadir airborne imagery and 2.5D urban modelling) and more advanced ones (e.g.oblique airborne imagery and 3D meshing) in order to deliver a final product that seeks to meet the needs of public administrations, real estate companies, security companies and general public.This process led to the development of a web-based geo-database management system, where both spatial data (i.e.building models reconstructed at various levels of detail) and associated non-spatial data (i.e.cadastral, lights and energy information) are stored, visualized, queried and manipulated.The content of the platform will be soon extended in order to include open data supplied by public administrations, further environmental pollution sources (e.g.CO 2 hot spots emission) and direct links to other publicly-available DBs.Additionally, the potentialities of the system will be enhanced through simulation-based services, i.e. further features will be provided like computation of energy heating demand, PV-suitability of the roofs and noise/nightlight pollution maps.Finally, the ultimate version of the service platform will be presented to the urban community of Trento, in order to create a dialogue among potential endusers and bring them from a common "nice-to-have" attitude to a more compelling "need-to-have" one.

Figure 3 .
Figure 3. Workflow adopted for modelling the building geometries (partially based on TU Delft -www.tudelft.nl)

Figure 4 .
Figure 4. Architecture adopted for the management of 3D geometries.

Figure 6
Figure 6 (top) shows an example of building reconstructed at different levels of detail from the Trento dataset.Starting from simple blocks model featuring prismatic buildings with flat roofs (left), geometry details are progressively enriched as roof structures (middle) and, finally, as wall structures (right).

Figure 6 .
Figure 6.Building models of Trento at various levels of details (top).3D mesh model of Graz textured with RGB and thermal orthoimages (bottom).

Figure 7 .
Figure 7.The current SENECA platform interface viewed at urban scale (left), neighborhood scale (middle) and building scale (right).

Table 1 .
Specifications of cameras and datasets adopted for geometry reconstruction.

Table 2 .
Results of aerial triangulation for the Trento and Graz datasets.