VISUALISING DETAILED CITYGML AND ADE AT THE BUILDING SCALE

There is an increasing activity in developing workflows and implementations to convert BIM data into CityGML. However, there are still not many platforms that are suitable to view and interact with the detailed information stored as a result of such conversions, especially if an Application Domain Extension (ADE) is involved to support additional information. We investigated the ease of use and features supported by visualisation software and tools with CityGML and ADE support, and propose an approach to develop a tool that combines useful features using a set of generic rules that can extract CityGML ADE attributes. The work, while generic, is geared towards detailed architectural datasets sourced from BIM. We implemented the approach in a web-based viewer supporting the visualisation of CityGML datasets enriched with ADE features.

Such conversions have been accomplished by different methods, some of which simplify and omit information from the BIM models to better suit CityGML and the geospatial world (Deng et al., 2016;Donkers et al., 2016;Kang and Hong, 2017;Lim et al., 2019). These methods sometimes extend the CityGML schema using Application Domain Extensions (ADEs) to accommodate the rich information found in BIM models (de Laat and van Berlo, 2011;. While such approach aids preserving the rich information, a downside is that CityGML datasets enriched with ADEs usually cannot be appropriately visualised and worked with, using the currently available CityGML software and tools. This paper has multiple goals. First, we provide a brief introduction to CityGML and ADE, as well as a review of software and tools available to support the visualisation of and interaction with CityGML files containing ADEs (both from the geometric and semantic aspect), focusing on their ease of use and the features they provide for interaction with objects in the model and their attributes. Since building-scale data may be heavy, we also include large files in the review.
Second, we discuss briefly the fact that there are few options that allow such interaction to be done easily and appropriately for these detailed building models obtained from BIM models, we suggest and demonstrate a possible approach to develop such a tool. This tool would ideally be able to open and visualise any CityGML model containing ADE attributes, support * Corresponding author at filip@nus.edu.sg viewing of all attributes linked to any selected object, and support filtering of the objects in view by any of the detected attributes, with the possibility of adding more features in the future. This would all be done with a minimum amount of setup, both for getting the platform ready and preparing the models for viewing, such that models can be viewed at the click of a button. The developed approach uses a combination of generic rules to extract information from both standard CityGML and ADEs, and by utilising a flexible web interface that has a wide support community. We hope that such a tool would be able to offer some support towards development and usage of other tools and ADEs at the building scale. Finally, we evaluate the limitations of our approach and briefly discuss other possible avenues for exploration in future work.

CITYGML AND ADE
CityGML is a data model for 3D city models standardised by the Open Geospatial Consortium (OGC). It is based on Geography Markup Language (GML) and it consists of a number of modules that support different aspects of the urban environment, including buildings, city furniture, vegetation, and terrain (Open Geospatial Consortium, 2012). One of the strong points of CityGML is its ability to store and communicate a large amount of semantic information in its models, in addition to visual and geometric information (Gröger and Plümer, 2012). For example, this includes information on surface types, heights, addresses, and names.
In addition to the wide range of semantic information it covers, CityGML features the Application Domain Extension (ADE) mechanism. These are additional, user-defined schemas that extend the standard CityGML to handle information aimed at specific domains. ADEs extend CityGML in different ways, e.g. geometrically (adding new geometric types), and semantically (adding new types of attributes and relations) (van den Brink et al., 2013;Kumar et al., 2018). A large number of ADEs defined for different uses have been created, documented, and researched . Some, such as the NoiseADE and EnergyADE (Czerwinski et al., 2006;Agugiaro et al., 2018), have become widely used and some have been recognized and added as additional thematic modules to the native CityGML schema, e.g. Bridge ADE with CityGML 2.0 and the Dynamizer ADE with the upcoming CityGML 3.0 (Chaturvedi and Kolbe, 2016;Kutzner et al., 2020).

REVIEWING CITYGML VISUALISATION AND ADE SUPPORT
There is a number of visualisation and geospatial software that support CityGML. However, many use cases involve using and visualising CityGML models by converting them to other data formats, since it is primarily intended as a data exchange model. This section reviews relevant software packages that are able to read CityGML data and visualise them either directly or through their own pipeline, without needing users to convert files separately using other software before viewing. Our review is focused on understanding their capabilities to read and display information provided through the ADE.

Methodology
We looked at seven software packages that are currently available, by opening and viewing six different datasets, focusing on how much of the files' geometry and properties the programs were able to support and view. This approach is similar to the methodology of Noardo et al. (2019), with our review more focused on ADEs. We also noted how easy it is to open a file for viewing and view the properties of the different objects in the model. Our selection of datasets includes a large CityGML model of Rotterdam in LOD1 (3D Geoinformation group, 2019), a NoiseADE sample dataset for buildings provided by OGC (Open Geospatial Consortium, 2012), an EnergyADE sample dataset of a house (KIT, 2017), and a UtilityNetworkADE sample dataset of the water pipes (Boates, 2019). The remaining two files were developed by our team, both sourced from a conversion from IFC -one containing indoor data in the standard CityGML form without an extension (Konde et al., 2018), and another of a large building containing an ADE. An overview of the files and their contents are shown in Table 1.

Software descriptions and observations
3.2.1 SpiderViewer SpiderViewer is a CityGML viewer developed by GEORES. It allows the customising of appearances and textures based on the semantic properties of CityGML elements. In our tests it was able to detect ADE schemas and namespaces within the files but was unable to resolve the schemas and thus was unable to display these models.
3.2.2 FZKViewer 5 One example of visualisation software that supports both CityGML and ADE is FZKViewer, developed by IAI/KIT, a tool for Windows that is able to visualise geometric and semantic information from IFC, gbXML, and CityGML files (KIT, 2020). In addition to the 3D view of the model, it is able to extract the hierarchy and relationships between objects in the file, as well as tables of properties or statistics regarding the objects found. Out of the box, it can view geometries and semantic information from some selected ADEs with built-in support, such as the Energy ADE. It is also able to display the geometry and a portion of the properties for the NoiseADE but was unable to for the UtilityNetworkADE. It allows the user to navigate through the hierarchy of the file and hide objects with a simple interface. It also allows for interaction with individual objects, like individual wall surfaces or windows. Selecting an object in the 3D view or in the attributes table highlights it in all other windows and pulls up a list of properties for the selected object.

azul
Azul is a tool for macOS that supports 3D visualisation of CityGML, CityJSON, OBJ, OFF and POLY files (Arroyo Ohori, 2020). It is able to open multiple files, and displays the attributes of objects when they are selected in the viewer or in the sidebar. It supports the IMGeo ADE and is able to extract geometries from any ADE for visualisation, but it has no full support for the extraction of ADE attributes. It was able to open files 1, 3, 5, and 6. With file 4, it was able to detect the geometry and list them in the sidebar, but no geometry was displayed in the viewer. With the files we used, some properties such as name and type were displayed but ADE attributes were not able to be extracted.
3.2.4 FME 2019.2 Another software package that supports CityGML and also comes with ADE functionalities is FME. It is a data integration tool and focuses on the conversion and translation of one format to another, supporting hundreds of different file formats and software, one of which is CityGML. It reads the XML file to find the ADE specifications where needed, or XSDs can be directly input to allow for ADE support. It comes with a visualising component, FME Data Inspector, which allows for the visualisation of information alongside the conversion process . It was able to open files 1, 2, 3 and 5 without any configurations or XSDs, with all NoiseADE properties from file 2 and EnergyADE properties and geometry from file 3. Files 4 and 6 require additional configuration in order to be opened.

QGIS 3.12
QGIS is an open-source geographic information system software that is able to load, view and edit CityGML files, among other supported file formats. When loading a CityGML file, objects from the different CityGML modules are detected and divided into different layers from which the user may select which ones they wish to import. It supports viewing the files in 3D, but this works only for certain types of files, while others may require pre-processing first. For example, while parts of the Rotterdam example are viewable in 3D, the NoiseADE example is only viewable in 2D. However, even though it is possible, we observed that viewing files in 3D can be taxing on the computer. ADE properties are fully imported and viewable in its property tables but ADE geometry may not be supported. For example, the FZKHouse example file's properties are all imported and can be viewed but the geometry is empty, while the Nanaimo water pipes file's properties and geometry are both supported.

CesiumJS and Cesium ion
CesiumJS is an opensource JavaScript library for creating 3D geospatial visualisations. It supports a large variety of 3D file formats, such as glTF, and also offers a separate service, Cesium ion, that converts various other file types, including CityGML, to 3D tiles and hosts them on their server for viewing (Amato, n.d.). Currently Cesium ion is focused on providing efficient tiling and other options like textures and clamping to terrain. It is able to extract properties and store them in batch tables to be used in CesiumJS viewers, such as for viewing when objects are selected, but it does not support ADEs at the moment. Due to the conversion process, it treats buildings as single objects, so it is not possible to retain and view properties for individual building elements using this service. Cesium ion is easy to use and converts and tiles selected files without any complicated configurations. Upon conversion, a link to a simple sandbox to preview the converted file is provided, but in order the view the models and its properties properly, customisation of this code or a separate CesiumJS implementation is required.
3.2.7 3DCityDB and 3DCityDB web-map-client Due to the large nature of urban-scale 3D models, a common way to manage CityGML datasets is the use of a database to store and manage the large amount of information. Often, in conjunction with this, is the visualisation of the information in a separate application or web-based interface, with models and data streamed as needed from a server. In such environments, the CityGML models are usually converted into other file formats with better visualisation support, and the attribute information, such as names, heights, and addresses, are extracted from the models and stored in a separate format to be queried as needed. However, the models themselves no longer carry the semantic information, which may be crucial for use cases. Such an approach has been researched with a wide variety of combinations between file formats and platforms such as JSON files with WebGL browser interfaces, custom C3D files with a Java applets, glTF with CesiumJS, and OBJ with mobile applications (Gesquière and Manin, 2012;Prandi et al., 2015;Simoes et al., 2015). A widely used database is 3DCityDB, an open source spatial relational database which employs either Oracle or PostgreSQL to store and manage the information found in CityGML files while retaining both semantic and geometric data (Yao et al., 2018). It provides a frontend interface which makes it easy to import CityGML files and convert and export information to KML, COLLADA, glTF and spreadsheets, automatically tiling the models in the process. It also offers a separate web client based on CesiumJS that can be configured to view the exported files in a web browser. In this web-mapclient, different sets of files can be organised into different layers, objects can be hidden and revealed, shadows and terrains can be toggled, and it can also be set up to display object attributes upon selection (Figure 1).
Since v4.0.0, it also has implemented ADE support into their database, in which XSDs can be used to configure the database to store the required information. However, the ADE functionality is not yet fully automatically integrated with their frontend interface and will require the user to develop the explicit mapping information themselves in order to configure the database and process the models for viewing (Yao et al., 2018). A large number of development and research efforts into CityGML visualisation focuses on how to best use 3DCityDB to perform and look better, and integrate with other information sources and types. This includes development of different ways to display models, integration with services to draw data from a wide variety of sources and many other works (Prieto et al., 2012;Pispidikis and Dimopoulou, 2016;Wang et al., 2018). Due to the conversion of files to glTF and KML, the 3DCityDB webmap-client is only able to select what 3DCityDB calls 'top-level elements'. This means that it can only select entire buildings as one object and is not able to allow selection of individual building elements. Aside from this main disadvantage, even though the 3DCityDB interface greatly simplifies the setting up and managing of CityGML files, it still requires an amount of additional work to set up, configure and export the files and spreadsheets for viewing. In addition, although the web client supports the viewing of attributes upon selection, the built-in setup is done such that it requires the spreadsheets to be made publicly accessible via Google Fusion Tables. It would require a separate implementation to support this attribute viewing feature differently.

Results and takeaways
The results of opening the six files in the seven visualisation software are summarised in Table 2, while Table 3 shows a summary of the features that are supported. The main takeaway from these simple tests with a number of different software and files was that CityGML viewers that support ADE fully out of the box are not common. The ones that are able to do so require setting up with the ADE XSDs and other additional configurations to properly handle files containing ADEs. Meanwhile, those that are easy to use are only able to support ADEs partially. In addition, the options that rely on conversion of the CityGML into other formats are only able to treat buildings as single items and selection of individual building elements is not supported. When such a system is used with the detailed LOD3 or LOD4 models of buildings at the building-scale, the rich information stored deeper in the model, such as at each storey or in each room, cannot be accessed. Such a shortcoming can be detrimental for certain situations and use cases.

Lessons learned and goals for our work
Our project aims to look into ways to support the visualisation and querying of CityGML objects at a building scale, inclusive of ADE support, in order to view such models in an easy and meaningful manner. In addition, we also hope to devise a method to view CityGML models based on the CityGML 3.0 schema and customised ADEs that are still under development and do not have stable XSDs. After examining the features of existing software and tools and the types of information found in CityGML models obtained from BIM models, we believe the ideal outcome would be to have a viewer with the ability to read and display files using generic rules that extract both geometry and properties, hopefully with results similar to QGIS where all the ADE properties are automatically detected. These rules would be loosely based on the CityGML schema but will not have to rely on the schemas for the ADEs, such that no configuration or stable XSDs are needed. The viewer would have to be able to allow the user to select individual building elements and view their properties, much like FZK Viewer, in order to properly capture the all the information stored. We also hope that the viewer would have the accessibility, interactivity and The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-4/W1-2020, 2020 3rd BIM/GIS Integration Workshop and 15th 3D GeoInfo Conference, 7-11 September 2020, London, UK  possibility of wider support for different information that the platforms created with 3DCityDB and CesiumJS offer. This would be done by detecting the native CityGML information in the file and extracting geometry and attributes according to the specified schema, while applying generic text content extraction for anything outside of the schema, which is assumed to be ADE information. In this way, the viewer does not need detailed information on the ADE and can be used to view ADEs still under development, or without XSD files. It could be a possible way to look at such files to get a better idea of what ADE properties are inside them, without having to look directly at the XML code.

METHODOLOGY
We have developed a viewer to demonstrate the implementation of a tool that would be able to read and display CityGML files with ADEs using three generic rules. This was done in the form of a web-based interface, using CesiumJS to support the visualisation.

CesiumJS and its Entity API
Apart from CesiumJS's support for a wide variety of file formats directly and through Cesium ion, it is also possible to create polygons and attach attributes to them on the fly, using its entity API. This allows us to read the CityGML files directly and create each polygon in the CityGML model separately. This would also allow each object or surface in the file to be viewed and selected as an individual object with its own semantic information, which is not possible with the other methods that use glTF or 3D Tiles. These include information such as its surface type and other attributes attached to the object, or information attached to objects higher in the tree such as information on the storey or building the object is located within. While not the most efficient solution in terms of performance, it is a simple method that could be used to view small files directly in the browser without the added hassle of setting up separate databases and servers or converting the file to other file formats.

Rule implementation process
The rule implementation process we have used is split into three parts. It aims to implement the rules in a way that will not require the entirety of the CityGML file to be checked against the schema, or keep too much of the file's hierarchy in memory while still allowing as much of the attributes to be extracted as possible. The first part is the identification of generic rules that would apply to a wide range of CityGML and ADE elements and allow a sufficient amount of information to be extracted properly with a small number of rules. The second part is the processing of the CityGML schema to derive a representation of these rules that could be used to inform the viewer of when each rule should be applied. Ideally, this would cover the entirety of the CityGML schema. The last is the development of a viewer that would apply these rules to the CityGML files it receives and extract the necessary information and display them in the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-4/W1-2020, 2020 3rd BIM/GIS Integration Workshop and 15th 3D GeoInfo Conference, 7-11 September 2020, London, UK  (3) Check for geometry and attributes.
The first rule is used for elements that would directly lead to only textual information that should be grouped and displayed together. This rule is assigned to elements such as xalAddress and is the default rule used. This is also the rule used for ADE elements. The second rule is used for elements that would directly lead to only geometrical information and no longer contain any attributes that would be useful to be viewed as plain text. Such elements include lod4MultiSurface and lod1Solid, as well as their children. The third rule is used when an element could contain both geometrical and textual information in its children nodes. As such, they need to be processed further to determine what elements are further down the branch and what rules to apply. This rule is assigned to any element that would eventually lead to elements with rule 2. The combination of these three rules would allow simple geometrical and textual information to be extracted from the CityGML file in an appropriate manner to be displayed in the implemented viewer. This is just a subset of all the cases present in the full CityGML schema and is limited in its ability to handle different types of ADE schemas but we found it sufficient for the purposes of this demonstration.

Representation of rules
After identifying the generic rules, the viewer would need specific instructions on when each rule should be applied. This was done by processing the CityGML XSDs to extract a JSON file of key-value pairs that instruct the viewer on what action to perform when a particular element is encountered in the CityGML file. The intention was to generate this file for any version of, or any subset of modules for CityGML as needed and specify it in the viewer. To achieve this aim, a recursive function is used to check each element in each of the relevant XSD files, following paths to all its possible substitution groups, types, references, extension bases and child nodes, and subsequently all possible paths from each one of those elements. When an element that matches a rule is found, a key-value pair is set and a value is returned to inform the determining of rules for parent elements as the function unwinds. Figure 2 illustrates this process using a flowchart.

Application of rules
We implemented the three rules from the first step, as well as some other rules that extracted information for positioning the models from detected EPSG codes, in the viewer. The rule application process is shown in Figure 3 using file 2 as an example.
On the left is an abstraction of the XML found in the file, which each element colour coded according its respective rule. Elements that should be processed with rule 1 are shown in light grey, rule 2 in black and rule 3 in dark grey. ADE elements which no rule matches are shown in white. The file is read starting from the cityModel element and it is checked against the JSON file generated during the second step to determine which rule should be used before moving on to its child elements. Matches for rule 1 are processed immediately. The element's and all subsequent child elements' values are all extracted and added to a javascript object as shown on the right. Elements that are unable to find a matching rule are also processed immediately and are added to a separate category for ADEs. This was done so we could better inspect the different properties in the file. Matches for rule 3 are processed next. Elements that match this rule are collected and checked after all its sibling elements have been checked. Newly extracted values are added to the javascript objects from before, creating new categories for each new hierarchy as required. Finally, matches for rule 2 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-4/W1-2020, 2020 3rd BIM/GIS Integration Workshop and 15th 3D GeoInfo Conference, 7-11 September 2020, London, UK are processed, after all the elements for rule 3 have been fully processed. The elements are processed to generate geometry and the most updated javascript object containing all the properties extracted is assigned to the Cesium entity created. In this example there was no geometry at the CityModel level but at the Building level this rule is used on lod1Solid, to generate the box on the right. The final result of this process is shown in Figure 4.

RESULTS AND EXAMPLES
The viewer we implemented was tested with the same six files used to review CityGML software in earlier sections of the paper. The files used show the features that the viewer is able to support as well a few limitations of the current implementation (akin to We used the existing code developed extract rules from the CityGML 2.0 schema to generate a new JSON file for the rules using a sample of the CityGML 3.0 XSD. Using this new JSON file, the viewer was able to properly display this sixth file as well. However, we realised that the way the various building elements were defined was different from earlier examples and thus the colour coding of the objects did not apply. We also realised that ADE elements were not able to be displayed properly, as they took the form of multiple elements with the same name.

DISCUSSION AND LIMITATIONS
Based on the observations and issues faced when trying to view different types of files, we identified some main limitations which sparked some ideas for future explorations and work.

Performance
One main limitation of the current demonstrative viewer is that it has poor memory performance and can only support models up to about 50MB in size. This is due to inefficient use of the entity API to create each polygon with its properties and keep the entire model in memory instead of using more efficient solutions like glTF models and 3D tiles that Cesium also supports. This choice was made to avoid the need to preprocess the CityGML files, as well as because of the lack of a simple and accessible method to convert CityGML to glTF object by object, rather than building by building, with common conversion methods requiring an intermediary file format like COLLADA and using the COLLADA2glTF converter (Khronos Group, 2019). However, it would be beneficial to further investigate methods that make use of these file formats to improve performance for larger files in the future. Some interesting directions could be to try adapting 3DCityDB and its web client to accommodate building objects and generic rules. It would also be interesting to look deeper into conversion to glTF that maintains some semantic information as glTF also has the capacity to represent a model as a collection of scenes and nodes with additional custom properties attached, although these are typically used for animation purposes and may not be appropriate (Khronos Group, 2020).

Rule coverage
Another limitation is that many cases are not supported in our current implementation. Currently the rules derived from the XSDs are focused on viewing simple files containing elements from the Building module and support for other modules and more complex files are not fully supported. This is due to the simple rules used to process the XSD and CityGML files. In CityGML there are more complex relationships such as xlink references and implicit geometry that will need to be considered to fully support all aspects of CityGML. In addition, all the ADE information is only treated as simple text content. Geometry or other types of information are not properly considered at the moment. Only the final value of the ADE nodes are extracted as well, so not all information from the ADE nodes are fully considered and extracted. It may be beneficial to implement proper class binding for the native CityGML schema, such as in other CityGML software, and more effort will need to be spent defining rules and their conditions to ensure that all objects are treated properly. An approach that fully tracks and analyses the paths found in the file in order to determine which rules to use or which properties are assigned to which object might be one way to do this.

CONCLUSION
CityGML usage and visualisation is now integrated with a variety of applications and information sources and it has been adopted in a number of domains and locations. ADEs are also developing at a fast pace, with many being used today for a wide range of purposes. BIM to CityGML conversion now also has a range of support for mapping and simplifying geometry and translating properties and attributes that could aid a variety of use cases. However, it may still be difficult to make proper use of all these new developments, due to the fact that support and coordination between the separate elements are still developing. There are only a few platforms that are able to access and interact with all the newly obtained information and utilise them in a useful manner. Often, many layers of conversion, data structures and platforms still have to be implemented in order to utilise the data efficiently. In this paper we have developed a methodology towards bridging these gaps, and demonstrated it by implementing a web-based viewer supporting the visualisation of CityGML ADE-enabled files. We believe that our work, while preliminary, will contribute to increasing the usability of datasets with ADEs, which otherwise can hardly be visualised. Although we have picked up some features and developed a demonstration of a simple platform that would be able to view the information in CityGML models with ADE attributes at a building scale, much work still has to be done in order to come up with a useful platform that would serve the multitude of integration problems such models aim to tackle. For future work, we plan to improve the performance of the viewer, increase its functionality (e.g. support new types of geometries that can be defined by an ADE), and support CityJSON, a JSON-based implementation of CityGML, which recently has started supporting extensions akin to CityGML ADEs (Ledoux et al., 2019) and is gaining software support (Vitalis et al., 2020).