THE NEXT GENERATION VEXCEL IMAGING 3D CITY MODELLING USING DIRECTLY GEOREFERENCED DATA

Since their first wide scale adoption after WWII, airborne cameras have continuously served the airborne mapping profession for over seven decades. During this time, they evolved to be meticulously designed, engineered, and most importantly built with the lens geometry to produce high precision mapping products. Mapping Standards have been well established around the globe for federal, state, and municipal governments with these cameras’ capabilities and limitations in mind. On the other hand, in the last two decades, service providers from around the world proved that direct georeferencing systems have been the best and most efficient approach for airborne mapping using large and medium format cameras, especially for 3D city modelling and corridor mapping applications. The state-of-the art today is thus to use a high-end large format camera or an oblique camera including a high-end mapping grade nadir image with an embedded direct georeferencing system to produce high-precision and high-resolution 3D city models without the need or expense of surveying extensive ground control points. This paper presents the results of the state-of-theart in airborne digital mapping cameras from Vexcel that are tightly integrated with a direct georeferencing solution from Trimble Applanix. Two Flights are presented in this paper, namely: 1) Gleisdorf Calibration and Accuracy Assessment flight, and 2) Graz Map Production Flight. 1. VEXCEL SENSOR TECHNOLOGY This Section is dedicated to presenting the Vexcel UltraCam Osprey 4.1 system design, characteristics, specifications, and hardware/software components. 1.1 UltraCam Osprey 4.1 The fourth generation of Vexcel’s camera technology was introduced with the UltraCam Osprey 4.1 in 2020, shown in Figure 1. It is also the first Vexcel camera to include CMOS optical sensors. The UltraCam Osprey 4.1 comprises a mapping-grade 280 Megapixel nadir camera and four 150 Megapixel oblique camera heads pointing forward, backward, right, and left as shown in Figure 2. The Nadir mapping-grade 280 Megapixel camera comprises two 150 Megapixel camera heads. All six camera heads are equipped with 150 mega pixel CMOS sensors. The nadir camera consists of two panchromatic sensors contributing to the large 280 mega pixel nadir frame and additional RGB and NIR camera heads. The pixel size of all detector arrays of the UltraCam Osprey is 3.76 μm by 3.76 μm. The focal lengths of all the camera heads in the system are chosen to end up with the Ground Sample Distance (GSD) of the oblique image center to be similar to the nadir camera GSD. The UltraCam Osprey 4.1 is, therefore, a professional high-end mapping-grade system that captures both nadir and oblique imagery from 5 different angles for efficient 3D urban mapping. ______________________________________________ * Corresponding author Figure 2 shows the image footprint for all nadir and oblique images at 5 cm GSD. The Osprey 4.1 technical specifications are listed in Table 1. The new CMOS sensor technology delivers stunning image quality. Additionally, the new software-based adaptive motion compensation (AMC) replaces the FMC technology by adding angular motion compensation to the forward motion compensation. The standard traditional FMC technology did not allow for handling image scale variation of oblique images in general. Therefore, AMC is a breakthrough technology, especially for the oblique imagery. Figure 1: UltraCam Osprey 4.1, nadir and oblique camera system by Vexcel Imaging GmbH, Graz, Austria. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2022 XXIV ISPRS Congress (2022 edition), 6–11 June 2022, Nice, France This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B1-2022-309-2022 | © Author(s) 2022. CC BY 4.0 License. 309 Table 1: UltraCam Osprey 4.1 Technical Specifications Figure 2: UltraCam Osprey 4.1 Image Footprint at 1065 m H_agl / 5cm Nadir GSD


VEXCEL SENSOR TECHNOLOGY
This Section is dedicated to presenting the Vexcel UltraCam Osprey 4.1 system design, characteristics, specifications, and hardware/software components.

UltraCam Osprey 4.1
The fourth generation of Vexcel's camera technology was introduced with the UltraCam Osprey 4.1 in 2020, shown in Figure 1. It is also the first Vexcel camera to include CMOS optical sensors. The UltraCam Osprey 4.1 comprises a mapping-grade 280 Megapixel nadir camera and four 150 Megapixel oblique camera heads pointing forward, backward, right, and left as shown in Figure 2.
The Nadir mapping-grade 280 Megapixel camera comprises two 150 Megapixel camera heads. All six camera heads are equipped with 150 mega pixel CMOS sensors. The nadir camera consists of two panchromatic sensors contributing to the large 280 mega pixel nadir frame and additional RGB and NIR camera heads.
The pixel size of all detector arrays of the UltraCam Osprey is 3.76 µm by 3.76 µm. The focal lengths of all the camera heads in the system are chosen to end up with the Ground Sample Distance (GSD) of the oblique image center to be similar to the nadir camera GSD.
The UltraCam Osprey 4.1 is, therefore, a professional high-end mapping-grade system that captures both nadir and oblique imagery from 5 different angles for efficient 3D urban mapping. ______________________________________________ * Corresponding author Figure 2 shows the image footprint for all nadir and oblique images at 5 cm GSD. The Osprey 4.1 technical specifications are listed in Table 1.
The new CMOS sensor technology delivers stunning image quality. Additionally, the new software-based adaptive motion compensation (AMC) replaces the FMC technology by adding angular motion compensation to the forward motion compensation. The standard traditional FMC technology did not allow for handling image scale variation of oblique images in general. Therefore, AMC is a breakthrough technology, especially for the oblique imagery.  Vexcel's photogrammetric production chainthe UltraMap Software Productprovides a seamless fully integrated airborne image postprocessing and map production suite of software modules including: • Aero-triangulation for nadir and oblique images The UltraMap Software Product reaps the benefits of the quality GNSS/inertial data acquired by the integrated Trimble AP Air GNSS/inertial direct georeferencing system to optimize the geometric processing. Additionally, UltraMap supports the adaptive motion compensation (AMC) for all 4 th generation cameras of the UltraCam sensor family.

UltraNav
UltraNav features state-of-the-art Trimble AP Air integrated Inertial/GNSS direct georeferencing technology by Trimble Applanix. The Trimble AP Air components are fully integrated in the UltraCam sensor head.
UltraNav manages camera parameter settings, exposure triggering, and camera mount stabilization.
Both pilot and operator displays are directly connected to the camera head for mission guidance and in-flight quality control. The Trimble AP Air products are available in different performance levels including Trimble AP 50 Air and AP 60 Air. Error! Reference source not found. lists the accuracy specifications for the two above mentioned systems.

SYSTEM CALIBRATION AND ACCURACY ASSESSMENT
The Osprey 4.1 system calibration is done at Vexcel Imaging in Graz, Austria. The system calibration includes: Lever Arm calibration

Gleisdorf Calibration and Accuracy Analysis Flight
The UltraCam Osprey 4.1 has been flown in Gleisdorf, Austria on August 25th, 2021 for system calibration and accuracy assessment purposes. The flight trajectory is shown in Figure 7 where two flight altitudes have been flown at approximately 1,100 m and 2,200 m above ground Level (AGL), respectively. The two flight altitudes have been chosen to produce 5cm GSD for the lower flight altitude and 10 cm GSD for the higher flight altitude. In this case a Trimble AP 50 Air was embedded in the UltraNav and used for direct georeferencing. The orange-colored triangles shown in Figure 7 represent the UltraCam airborne image exposure locations. At each exposure location, all nadir and oblique images are captured synchronously. An example of the footprint of one exposure at the 5 cm GSD is shown in Figure 2. A total of twelve flight lines were flown at 80% forward overlap and 60% side-lap for the nadir camera as shown in Figure 8. Applanix POSPac MMS was used to process the raw GNSS/Inertial data to generate the Exterior Orientation for each image.
Instead of using a dedicated GNSS base station to augment the GNSS processing in POSPac to produce cm level accuracy, the POSPac Trimble Centerpoint Post-processed RTX (PP-RTX) service was used. PP-RTX uses corrections derived from a global network of stations operated by Trimble to achieve cm level position accuracy without the need for local base station data.
The corrections are automatically available to POSPac MMS via an internet connection within minutes after the survey.

The primary output from POSPac MMS is a Smoothed Best
Estimate of Trajectory (SBET) together with the performance metrics which are then used to generate the Exterior Orientation (EO) Parameters for each image at the exact exposure time. Figure 4 shows the estimated GNSS/inertial positioning accuracy while Figure 5 shows the estimated orientation accuracy. Note that the final orientation accuracy is better than the system specifications listed in Error! Reference source not found.. The forward/reverse separation of the PP-RTX GNSS position solution is shown in Figure 6. Note that the forward/reverse separation is a recommended QC plot to inspect in POSPac to assess the difference between processing the data forward versus backward in time. The consistency between the forward and reverse solutions indicates that the absolute accuracy of the trajectory is in balance with the RMS of the forward/reverse separation. In this flight, for example, the forward/reverse separation RMS is better than 1 cm for all positioning components while the maximum error occurred at the Height component of about 5 cm as shown in Figure 6, which is well within the RMS system specifications listed for the AP 50 Air in Error! Reference source not found.. These results are typical nowadays with PP-RTX and the Trimble survey grade GNSS systems that include all satellites and frequencies from GPS, GLONASS, BeiDou, and Galileo satellite positioning and navigation systems.
Statistically, POSPac positioning and orientation results shown in Figure 4, Figure 5, and Figure 6 indicate that the final ground object accuracy should be within one GSD.
Since the flight contained two GSD's; namely 5 cm and 10 cm, the lower resolution GSD is safe to be considered the overall project GSD for accuracy analysis purposes.

System Calibration Using the Gleisdorf Flight for Calibration and Accuracy Analysis
The SBET file has been used together with the imagery in Applanix CalQC Calibration and Quality Control software to calibrate the boresight angles and possibly calibrate the camera interior orientation if need be. However, in the case of the UltraCam product line, the camera calibration parameters are typically locked, since Vexcel's terrestrial calibration facility/process has been proven to provide metric quality accurate and stable camera calibration results over the years.
Therefore, in CalQC, only IMU/Camera boresight angles were calibrated, while the camera interior geometry has been locked as constant error free values. CalQC uses a relative bundle adjustment to extract boresight angles without the need of ground control. It starts with automatically generating tie points using its ATG engine. Then, it combines POSPac-computed EO parameters with those tie points and the camera interior geometry in order to extract the boresight angles.
Error! Reference source not found. lists the calibration results and their associated accuracy. It depicts that the boresight angular accuracy is well within the measurement noise of the IMU in use (listed in Error! Reference source not found.

System Accuracy Assessment Using the Gleisdorf Flight for Calibration and Accuracy Analysis
CalQC is typically used to also assess the check point accuracy if the calibration field has any ground control. Table 2 lists the check point residuals for the Graz Calibration and Accuracy Assessment flight. Please note that RMS (pixel) is computed using a 10 cm GSD. One Ground Control Point has been used in the processing workflow to calibrate and remove the global datum shift from the data. This typically results in good quality mapping products for urban mapping, especially ortho generation. It also allows for proper overlap of the along-track and the across-track oblique imagery coverage.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France Figure 9 shows the Graz Map Production Flight layout. Figure 10 shows the local image scale (of variable GSD) for nadir and obliques imagery at 5 cm nadir GSD.

Accuracy Assessment of the Graz Flight
Before producing any mapping products, the accuracy of the POSPac-computed EO was assessed against the 27 Check Points available in the mapping area. These check points are well distributed in the mapping area as shown in Figure 11.
The results of the accuracy assessment process are divided into two main sections. The first section addresses the nadir camera only while the second section addresses both nadir and oblique cameras in one AT process.

Accuracy Assessment of Nadir Imagery Only:
The Nadir imagery with EO were used independently in UltraMap to assess the check point accuracy using the 27 checkpoints shown in Figure 11. This was done twice: once using the POSPac-computed EO with 15 GCP's, and once using the POSPac-computed EO with no GCP's. The accuracy is expressed in terms of check point residuals. This fact has been well documented in the past couple of decades (c.f., Colomina and Blazquez, 2014, Ip et al 2007, Casella et al 2006, Ip et al, 2006, and Mostafa and Schwarz, 2001.)

Accuracy Assessment of Nadir + Oblique Imagery Together:
The nadir and oblique imagery were processed together in UltraMap using the POSPac-generated EO and evaluated against the 27 checkpoints shown in Figure 11.
As was done with the nadir only imagery, two assessments were done: the first using 15 GCPs together with the POSPac-calculated Exterior Orientation (EO) parameters, Error! Reference source not found. confirms that when accurate direct georeferencing is used with such a highperformance high quality airborne camera as the Vexcel Osprey, there is no need for ground control points in the AT process even in the presence of oblique imagery. Furthermore, the results show that pixel level ground accuracy is achieved using POSPac Trimble CenterPoint Post-processed RTX without the need for any local base stations. This means high accuracy mapping can be done without the need for any local infrastructure (base stations ore GCP's) except for the purposes of quality control (ie a few checkpoints to assess accuracy and one GCP to resolve datum). Table 6: Accuracy Assessment for Graz Map Production Flight using Nadir and Oblique Imagery Please note that a consistent global datum shift was present in both nadir stand-alone data and in the nadir + oblique data. This datum shift is typical and is easily calibrated and removed from the data using one GCP.

Mapping Product Generation for the Graz Map
Production Flight Once the system accuracy was properly assessed, map products were generated. A dense DSM was extracted (shown in Figure 12) using the accurate POSPac-computed EO parameters and the UltraMap-generated dense/accurate tie points. Using this DSM, along with the EO parameters, UltraMap was then used to generate an orthomosaic shown in Figure 13.
UltraMap made use of the dense and accurate tie points during the color balancing process that ensured radiometric uniformity of the orthomosaic. Figure 14 shows the UltraMap-extracted color-coded DSM (left) and its associated orthomosaic (right).

3D CITY MODELLING
3D City Modelling is a mapping product that is rather more modern than the standard classical photogrammetric mapping products which traditionally include a DEM and orthomosaics. 3D city models allow many professionals to build and maintain infrastructure in urban areas around the globe. One of the many advantages of the UltraCam Osprey design with its five viewing angles, is constructing a precise and comprehensive 3D model. In other words, when ground objects are captured by nadir and oblique imagery in successive camera locations, different lines of sight allow for reproducing a comprehensive 3D object geometry.
This is particularly true in the presence of accurate exterior orientation parameters produced by the Applanix direct georeferencing system. Figure 15 shows the 3D city model produced using the Graz map production flight in the UltraMap environment while Figure 16 and Figure 17 show a bird's eye view of parts of the 3D city model.

SUMMARY AND CONCLUSIONS
In this paper, the next generation Vexcel Imaging 3D city modeling system is introduced. UltraCam Osprey 4.1 is a fully integrated airborne professional-grade system that comprises a mapping-grade 280 Megapixel nadir camera, four 150 Megapixel oblique cameras, and a Trimble AP 50 Air direct georeferencing system. The system design and specifications are presented, as well as the results from the analysis of two flights flown in 2021. These flights, the Gleisdorf Calibration and Accuracy Assessment flight and the Graz Map Production flight, have been used to assess its accuracy.
System calibration results using the Gleisdorf Calibration flight show that boresight calibration accuracy can be obtained to a level that enables high precision mapping using the UltraCam Osprey 4.1. The boresight calibration accuracy (0.002 deg, 0.002 deg, and 0.007 deg) achieved was well within the embedded AP 50 Air system specifications. Ground accuracy after calibration was evaluated through an accuracy analysis exercise where 9 checkpoints were assessed and their residual RMS error was 0.4, 0.2, and 0.7 pixels respectively.
The Graz Map Production flight was flown to demonstrate the system performance in a map production environment. First, the system ground accuracy was evaluated using 27 check points. When using 15 GCPs together with the POSPac-computed EO for the nadir imagery in UltraMap AT environment, an accuracy of 0.3, 0.4, and 0.4 pixels was achieved for X, Y, and Z respectively. In comparison, using the same data without ground control points resulted in an accuracy of 0.2, 0.4, and 0.4 pixels for X, Y, and Z respectively.
This confirms that the nadir imagery from the Vexcel Osprey when directly georeferenced using the embedded Trimble AP 50 Air-and POSPac -Trimble Centerpoint Post-processed RTX (PP-RTX) is sufficient for traditional nadir based high precision mapping without the need for ground control or local base stations. One GCP is typically used to calibrate the global datum shift and remove it from the data.
When all nadir and oblique imagery were used in UltraMap AT together with POSPac-computed EO and 15 GCPs, an accuracy of 0.9, 1.2, and 1.1 pixels was achieved for X, Y, and Z respectively. Without using any GCP's, an accuracy of 1.0, 1.0, and 1.2 pixels was achieved for X, Y, and Z respectively.
It is clear from the results presented here, that the presence of GCPs does not add any value to the mapping process accuracy because of the already sufficient accuracy of the EO produced using the embedded Trimble AP 50 Air system in the UltraCam Osprey 4.1 and POSPac PP-RTX. Note that, typically one GCP is required to resolve datum shift and remove it from the data.
The UltraMap software was used to extract a dense DSM using the already measured tie points that are sufficiently dense and accurate in nature.
Using this DSM along with the exterior orientation parameters, UltraMap was used to generate an orthomosaic that is shown in this paper. The dense and accurate tie points were made use of during the color balancing process that ensured radiometric uniformity of the orthomosaic. Using Vexcel 3D modelling technology, a high-resolution 3D model of the mapping area in Graz, Austria, was produced leveraging all the viewing angles and lines of sight for the nadir and oblique imagery acquired from different locations along each flight line and from the neighboring flight lines. This resulted in the perfect geometrical environment for 3D object reconstruction that resulted in a high-resolution 3D city model with ground accuracy at the pixel level.