ACCURACY VALIDATION OF TILTED CAMERA SETUPS IN OPEN SKIES PROJECT AND MAPPING APPLICATIONS

Signed March 24, 1992, the Open Skies Treaty permits each state-party to conduct short-notice, unarmed, reconnaissance flights over the others' entire territories to collect data on military forces and activities. Observation aircraft equipped with sensors shall enable the observing party to identify significant military equipment. The Open Skies Treaty agreed on an observation of 30cm GSD. Based on 8 mid-format cameras for 3 flight levels, a system was adjusted to comply with the regulations. However, the overall aim was to also use the system for mapping purposes in Romania, specifically the medium altitude configuration. From this medium altitude configuration, one specific combination raised our interest. The designed wide corridor mapping system, using two RGB tilted cameras and one RGB nadir camera generates a certain range of resolution of the sideward looking tilted cameras and a non-homogenous distribution of the GSD in the overlapping areas. While a reduction of the GSD in the remote parts of the tilted cameras is a well-known and accepted fact, the effect in the overlap of the tilted cameras with the nadir one is the opposite mathematically. In some cases, such an effect can cause a better GSD in these areas than expected. 1. THE OPEN SKIES TREATY On March 24 1992, in Helsinki, Finland, the idea of the United States President Dwight D. Eisenhower becomes reality after about 37 years. The original concept, intended to be a bilateral agreement between the United States and the Soviet Union, due to the Canadian Prime Minister Bryan Mulroney and Canadian Secretary of State for External Affairs Joe Clark, has initial turned into a 25 nations aerial observation treaty. Although nowadays satellite technologies can collect images with a higher resolution, aerial observations conducted under the force of the Open Skies Treaty gives access at information hard to obtain by all the treaty states-parties and build up confidence between them. Table 1. Comparison between Open Skies and Photogrammetric terms The most important aspect during Open Skies missions is to keep a specific AGL during the observation flights in order not to obtain a better ground resolved distance (GRD) than 30cm referring to the highest point into a leg. An Open Skies mission is defined by segments, legs, observation periods and event IDs. _____________________ * Corresponding author To harmonize the photogrammetric concepts into the Open Skies terms, there was a series of technical decisions that states the terms definitions. For this, terms that define an Open Skies mission can be outlined using notions from photogrammetric mission planning (Table 1). Even if, there are a lot of similarities with the photogrammetric domain, specific rules are mandatory to be followed by any state member that wants to have a certified platform for Open Skies missions. Thus, to integrate and adapt a commercially available photogrammetric system to fulfill the treaty requirements is always a challenge. Starting in January 2019, various Romanian governmental institutions started their search for a digital aerial system that could replace the analog aerial system certified in 2008. A Phase One 8 mid-format cameras system, integrated by GGS Germany, was chosen after several technical meetings. Figure 1. RoDAS Team and the AN30 Observation aircraft Open Skies terms Photogrammetric terms Mission number Flight project


THE OPEN SKIES TREATY
On March 24 1992, in Helsinki, Finland, the idea of the United States President Dwight D. Eisenhower becomes reality after about 37 years. The original concept, intended to be a bilateral agreement between the United States and the Soviet Union, due to the Canadian Prime Minister Bryan Mulroney and Canadian Secretary of State for External Affairs Joe Clark, has initial turned into a 25 nations aerial observation treaty. Although nowadays satellite technologies can collect images with a higher resolution, aerial observations conducted under the force of the Open Skies Treaty gives access at information hard to obtain by all the treaty states-parties and build up confidence between them.

Table 1. Comparison between Open Skies and Photogrammetric terms
The most important aspect during Open Skies missions is to keep a specific AGL during the observation flights in order not to obtain a better ground resolved distance (GRD) than 30cm referring to the highest point into a leg. An Open Skies mission is defined by segments, legs, observation periods and event IDs.

_____________________ * Corresponding author
To harmonize the photogrammetric concepts into the Open Skies terms, there was a series of technical decisions that states the terms definitions. For this, terms that define an Open Skies mission can be outlined using notions from photogrammetric mission planning (Table 1).
Even if, there are a lot of similarities with the photogrammetric domain, specific rules are mandatory to be followed by any state member that wants to have a certified platform for Open Skies missions. Thus, to integrate and adapt a commercially available photogrammetric system to fulfill the treaty requirements is always a challenge. Starting in January 2019, various Romanian governmental institutions started their search for a digital aerial system that could replace the analog aerial system certified in 2008. A Phase One 8 mid-format cameras system, integrated by GGS Germany, was chosen after several technical meetings.

The missions at different altitudes
To increase the success of an Open Skies mission, choosing the right height configuration is crucial as within the nations are allowed to perform an observation flight with a time frame of 96 hours -unless a demonstration flights is required, which allows for a further 24 hours required.
Hence, the various forms of relief of the observed countries, weather conditions for long periods of time and traffic restrictions over some areas need to be considered. Furthermore, aircraft technical specifications, such autonomy and flight height limits, must also be considered.
To adapt to the relief conditions, the height configuration must permit crossing over all kind of landforms but must not exceed aircraft height limits. Also, flying below the clouds with a large swath, to observe as much as possible of the area of interest, is a must for Open Skies missions. Hence it is crucial to also consider the ability to avoid traffic restrictions when choosing an Open Skies aerial system. Due to these factors, we chose three theoretical height configurations for the Open Skies missions: a. "Low configuration" consisting of two iXM-RS100F cameras (1 RGB and 1 Achromatic NIR) with a focal length of 23 mm which results in aHtheoretic of about 1564m, designed to be a "below clouds configuration.
b. "Medium configuration" consisting of four iXM-RS150F cameras (1 Nadir RGB, 1 Right RGB -looking left, 1 Left RGB -looking right and 1 Achromatic NIR with a focal length of 40 mm which results in a Htheoretic of about 3340m, designed to be an "optimal configuration"; c. "High configuration" consisting of two iXM-RS150F cameras (1 RGB and 1 Achromatic NIR) with a focal length of 70 mm which results in a Htheoretic of about 5570m, designed to be a "restrictions avoidance configuration".
The "optimal configuration" design for Open Skies missions is also a complex and an optimized aerial system for photogrammetric purposes. It is a custom version of similar commercial digital aerial systems on the market like Leica CityMapper, Vexcel Osprey 4.1 or Phase One's PAS880, with the mention that it can be used for both kinds of aerial missions, making it rather unique.

Camera setup
The Open Skies Treaty agreed on an observation of 30cm GSD at different flight altitudes has been agreed. Thereby, a system was designed for the Romanian air force supporting three flight levels. Based on 8 mid-format cameras, a system was adjusted to comply with the treaty rules and regulations. However, the overall aim was to also use the system for mapping purposes in Romania, specifically the medium altitude configuration consisting of four multispectral cameras (2 RGB tilted cameras, 1 RGB nadir camera and 1 Achromatic NIR enabled nadir camera). The low configuration -for a flight altitude be-tween 1100-1600m-uses two Phase One iXM-RS 100F cameras with 23mm lenses. These 100MPix cameras (one RGB, one Achromatic for NIR) generates a 4-band image with a GSD of 30cm at 1500m AGL. The opening angle of the 2 cameras is 98/82°. The real coverage RGB/NIR is 3.482m along track and 2.612m cross track.
The high configuration -for a flight altitude between 5500-6000m -uses two Phase One iXM-RS 150F cameras with 70mm lenses. These 150MPix cameras (one RGB, one Achromatic for NIR) generates a 4-band image with a GSD of 30cm at 5.585m AGL. The opening angle of the 2 cameras is 42/32°. The real coverage RGB/NIR is 4.261m along track and 3.196m cross track. Our interest was specifically raised the medium altitude configuration. The designed wide corridor mapping system, using 2 RGB tilted cameras and 1 RGB nadir camera generates a certain range of resolution of the sideward looking tilted cameras and an non-homogenous distribution of the GSD in the overlapping areas. While a reduction of the GSD in the remote parts of the tilted cameras is a well-known and accepted fact, the effect in the overlap of the tilted cameras with the nadir one is the opposite mathematically. The medium configuration -for a flight altitude between 3.000-3.500 muses two Phase One iXM-RS 150F cameras with 40 mm lenses. This 150MPix cameras (one RGB one Achromatic for NIR) generates a 4-band image with a GSD of 30cm at 3.191m AGL. The opening angle of the 2 cameras is 67/53°. The real coverage RGB/NIR is 4.261m along track and 3.196m cross track. In this study the two additional two Phase One iXM-RS 150F cameras with 40mm lenses were mounted oblique in portrait format while the nadir ones are in landscape orientation.
The entire system is designed in camera groups for each flight level. The full wiring plan is shown in figure 2.

Certification procedures
To be able to certify an aerial platform for Open Skies missions, a state member shall follow the schedule presented in figure 3. As a first step for certification, a series of data gathering sessions must be conducted, to obtain a set of Hmin-expected **established from the flight test data and provided before certification. Based on the aircraft and technical systems specifications, together with the results obtained for Hmin-expected, the next step is to fill in and submit the "Certification Technical Document" that represents *a data package of technical information pertaining to an observation aircraft and its associated sensors. Until the "Pre-certification event", a "Certification simulation" can be performed, to train the crew and resolve possible issues that may appear during the final event.
Following, a "Pre-certification event", that represents an international data gathering event hosted by a Certifying State party to collect flight test data and demonstrate certification procedures, processes, and technical capabilities of a sensor configuration intended to be certified shall take place, followed by an "Intermediate Co-ordination Meeting (IWGC)" which is *a meeting convened by the OSCC Informal Working Group on Certification to provide an opportunity for State parties to discuss questions concerning the forthcoming certification event.
Finally, the "Certification event" will take place, after which a platform will or will not be accepted for future Open Skies missions. If the platform can be certified, a Hmin-flight will be established, representing the value of the minimum height above ground at which each sensor configuration may be op-erated in the case of an observation flight following a demonstration flight. If it cannot be certified at this stage, all limitations need to be resolved and new "Certification events" shall be organized until all the Open Skies requirements are fulfilled.

MID CONFIGURATION WITH OBLIQUE CAM-ERAS
The mounting of the four cameras for the mid-altitude missions is flexible. While the two Nadir cameras, the RGB and the NIR are mounted fixed, the oblique cameras can change the tilt angle in a range between 20 and 45 degrees and change the orientation from landscape to portrait. While accepting a GSD range within the oblique camera setting results in a significant lower GSD at the outer parts of an image, it can nevertheless generate wide corridor mappings or 3D city modelling in urban areas for civil applications. urban areas.
For open Skies Missions the two sideward looking cameras are intended to be mounted, if the aircraft permits with a 44-degree oblique angle, which results in the footprint shown in figure 4: There are several advantages capturing data in sideward for mapping purposes and also for the Open Skies mission, where the swath width is significantly increased. As outlined in the calculated footprint in figure 4, the GSD in the overlap zone with the nadir camera increases. This has a positive effect on capturing 3D data for mapping. The footprint becomes a trapezoid.

Theoretical GSD distribution
The graphic in figure 5 was calculated by standard trigonometric equations implementing a rotation matrix to allow for flexibility for all rotation axes. This effect is based on the nonparallel orientation of the terrain related to the image. In normal nadir flights, the relation between the terrain distance and the nodal point, and the nodal point and the sensor plane is the same all over the image, not taking terrain variation or lensdistortion into account. The GSD was calculated by using the closest and remotest line-length and number of pixels. This defines the range in a best approximation, however a rectangle captured with an oblique setting also results in a trapezoid. The mapping mission performed was planned at 15cm GSD and the rotation of the oblique cameras was set to portrait as shown in figure 6. The tilt angle was set to 24 degrees, the final rotation after system-calibration was calculated in 23.960 degree for the left viewing camera, 0.212 degrees for the nadir camera and 23.743 degrees for the right camera.

Analytics on target validation
After performing observation flights over a calibration target to determine Hmin-expected, there are a series of reading sessions. During these sessions, the OSDDEFs image targets are analyzed by specially trained personnel, to identify the minimum resolved group bars, that in fact represents a GRD (ground resolved distance) determination. Due to the fact, that GRD is an empirical concept and not a theoretical one like GSD (ground sample distance), its determination is subjective. For this, the GRD depends on the inspector's experience, image adjustments performed, atmospheric conditions during observation flights, monitor settings, target modulation, lens distortions etc. To limit the effects of human subjectivism a large data set is analyzed, accompanied by a set of rules that need to be followed during the reading sessions, such as: limitation of flight time only on good weather conditions, gamma parameter applied to the OSDDEF images should be set to 1, optimization of image radiometry (contrast and brightness), target modulation should be in a specific range etc.
Thus, an "accurate" GRD is determined for each image analyzed and the final GRD is obtained as an average of all the intermediate ones. GRD is an important concept in Open Skies missions, because defines the Hmin-expected. However, GRD can be also important in photogrammetry, where usually GSD is the main parameter considered. For example, when planning a flight project at a particular GSD, if camera settings are not applied correctly, weather conditions and atmospheric illumination are not taken into consideration or other procedural issues occur, the GRD obtained will be worse, hence influencing the image quality. While GSD is a theoretical concept, GRD is the real parameter obtained during photogrammetric missions that defines the real image quality.
A particular case of a GRD determination was conducted over the three RGB cameras from the medium configuration of the Romanian Open Sky system. The main goal was to practically determine the location of the best resolution (GRD) on the nadir and on the 23 o tilted images, by performing several test The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B1-2021XXIV ISPRS Congress (2021 flights over the Open Skies calibration target. For this, the registered images were split in a 9-cell grid as outlined in figure  7. Figure 7. The 9-cell grid used for GRD determination After executing the flights, a data set of images for all three RGB cameras was obtained, containing the calibration target in every cell of the grid. Subsequently, by inspecting the data set, the findings concluded that when preserving the focal length of the cameras but changing the tilt of the cameras with a specific angle, the location of the best resolution (GRD) on the nadir camera is located in the image PPS (image middle) in the cell number 5 of the grid, while for the tilted cameras the location of the best resolution (GRD) is in the cell number 4 or 6, depending on the camera position (figure 8).

Figure 8.
Right RGB camera -left looking /Nadir RGB camera/Left RGB camera -right looking

Calibration over image block
A mapping mission with a GSD of 15cm was flown over the city of Constanta at the shore of the black sea. The area is relatively flat and the height calibration was performed by the use of point matching of all cameras and urban structure with high buildings. All cameras use the same type of sensor and lens -hence a valid calibration result could be expected. As well as the calibration of the focal length, the radial distortion and the offset parameters were calibrated. The results are shown in table 2, and the results are very similar.

Geospatial resolution validation
The real spatial resolution was performed measuring image coordinates in a GIS environment. For this, a rectangular object in Constanta was selected that was visible in 27 images. The location was analyzed by vectorizing the object boundary and calculating the perimeter and the object's size in pixel values.

Table 3. Resolution on pixel-based object analytics
This method results in different information compared to the GSD that was calculated before on the closest and remotest baseline. While the perimeter takes all four sides into account, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B1-2021XXIV ISPRS Congress (2021 if it is a rectangle or a trapezoid in the image, or in some cases a rhombus, the area can be seen as a integral value. However, the extremes are clearly visible in both. The position was estimated, and the perimeter and area calculated by the manually digitized polygon. Besides the radial distortion and variants in the flight, the drawing of the polygon cannot be seen as 100% exact. Still, the result is highly significant.

Figure 9.
Up the target area measured in the nadir_3410 image, in the middle the higher resolution on image left_2955 and below the image right_3087 with the lowest resolution.
All nadir values show a very similar size and perimeter, independent from their location on the images -this was what we As the other extreme, the resolution drops to 30% at the remote parts compared to the nadir images -that is about 60% by perimeter, which relates to 72% on the line-based calculation.

OTHER OBLIQUE CAMERA SETUPS
So, can this camera setup also be used for 3D city models? Yes, however there are some limitations. For example, focal length is very short and the range of GSD in the obliques very high. The higher resolution in the close nadir part does not really support the model and the facades typically need a close to the nadir average GSD. The 3D model created out of the calibration flight data is less accurate than other models, mainly due to the low GSD of 15cm. However, the multi views of a wider angled lens prevent greater gaps in the model. By analyzing other camera models, we established that the optimal relation between nadir and oblique cameras is a similar GSD for a wider part of the captured area. The longer focal length results in a smoother GSD distribution. The closer opening angle can cause less perspectives for the model due to hidden areas and a more difficult aero triangulation due to the distance of the corresponding images being wider. An additional disadvantage is the increasing number of flight lines and images with longer focus. A typical combination are Oblique imaging systems that have a slightly longer focal length in the oblique cameras than the nadir. A common range is 1.2-1.4 e.g. Nadir 90mm and oblique 110mm. This is a very common combination. Just looking on the aero triangulation. A very flexible system is the Oblique Imaging Systems of GGS that enable different lens combinations and camera orientations. The Phase One PAS880 uses in Nadir 90mm and Oblique 150mm which gives a 60% higher resolution in the oblique data than in the Nadir and enables a very high GSD even from a higher altitude.

RESULTS, PROS AND CONS
The validation of the camera setup, analyzed in this study gives a higher resolution in the close nadir area and a strong reduction of the resolution in the remote area. This is an expected result which was analyzed in detail in this paper. The use for 3D city models is possible but limited according to the resolution. However, this combination generates a better result in the overlapping area while the redundancy gives a higher reliability e.g. of the 3D point cloud. This setup is a very unique one and besides the oblique views it benefits in typical 3D mapping applications with very good accuracy for height measurements. For Open Skies mission the oblique configuration gives the advantage to obtain a large swath on the ground, that cannot be obtained only by using a single nadir camera, which results in a higher productivity and reduces the mission time and costs. This setup is very different to other oblique setups and has its benefits in the ability to cater for both mission types. By being a flexible system, the 8 mid-format PhaseOne cameras designed by GGS, it is a reliable system that can be used with success in mapping missions, but also fulfills the requirements of the Open Skies Treaty. The use of oblique data has become the new standard within the last five years. In many cases it is used as a add-on for information for homeland security, disaster prevention, city planning and other applications, particularly were the resolution is less important that the ability to identify the urban structure. However, a calibrated and well-balanced oblique camera system delivers both, better information and better geometric resolution and accuracy. The need for additional and longer flight lines is a budget factor on one side and for data handling on the other. Being aware that a single shot with the Open Skies setup is 450MPix, with the PAS880 880MPIX and the OISXL++ rather than 1 GPIX, storage and processing is the new challenge for this type of data acquisition.