PRELIMINARY INVESTIGATION INTO THE GEOMETRIC CALIBRATION OF THE MICASENSE REDEDGE-M MULTISPECTRAL CAMERA

Multispectral cameras, in the past the prerogative of Remote Sensing (RS) applications via satellites and manned aircraft, are becoming increasingly used in photogrammetric applications. Moreover, the ubiquitous use of Unmanned Aerial Vehicles (UAVs) has created a need for the miniaturisation of sensors, which has contributed to the availability of a wide range of relatively low-cost and lightweight cameras. Therefore, small multispectral cameras mounted on UAVs provide an effective and low-cost solution when it comes to acquiring airborne radiometric data. With the growing interest for such sensors to perform photogrammetric tasks, camera calibration remains an essential step in order to obtain reliable and geometrically accurate information. This paper will investigate the camera calibration parameters between the five bands of the MicaSense RedEdge-M sensor from laboratory trials. The results of the camera calibration will be obtained from the use, primarily, of Australis software and a calibration frame within the Nottingham Geospatial Institute. The variations of the parameters demonstrate the need for distortion correction separately within each band before using the images for photogrammetry.


Background
Multispectral sensors are largely used in RS applications. They are capable of recording different bands of the electromagnetic spectrum, being sensitive to different wavelengths through the use of different lens cones or filters. The employment of such cameras for photogrammetric applications is becoming of increasing interest. Moreover, the ubiquitous use of Unmanned Aerial Vehicles has created a need for the miniaturisation of sensors, which has contributed to a wide range of relatively lowcost and lightweight cameras now available on the market. Therefore, multispectral sensors provide an effective and lowcost solution when it comes to acquiring data for various applications. Among these, vegetation mapping (Laliberte et al., 2011;Kelcey and Lucieer, 2012) and precision agriculture (Honkavaara et al., 2012;Zarco-Tejada et al., 2013) are probably the most common.
It is well known that, differently from metric cameras, sensors not specifically designed for photogrammetry lack geometric stability. For this reason, a rigorous camera calibration is, now more than ever, a frequent vital step in the photogrammetric process. Attention must be paid to the camera calibration parameters that might change due to mechanical stress (during the physical use of the camera), temperature changes and hardware ageing (Bychkovskiy et al., 2003). Previous research has demonstrated that, in order to obtain high-quality results, there is a need for a robust recovery of camera calibration parameters as the first step in the photogrammetric workflow. Camera calibration can be defined as the process of measuring the relationship of a 'real' camera geometry in comparison to * Corresponding author perspective geometry (Smith et al., 2007), performed to model the deviation between the ideal mathematical model of central perspective and the physical reality of the camera (Luhmann et al., 2016).
Calibration procedures are widely implemented within most photogrammetric software commonly used to process the collected imagery. Amongst the various algorithms introduced to perform automatic calibration, one of the most popular is the well-known self-calibrating bundle adjustment, which represents now a standard and routinely applied operation (Barazzetti et al., 2011). Recently, also the Computer Vision community has shown an interest in the topic. Yet, their focus is often more on the development of user-friendly and fully automated procedures rather than on the calibration algorithm development.
The key features of a typical photogrammetric network for camera calibration can be found in the majority of papers related to this topic (see, for example, Brown, 1989;Fraser, 2006, Fraser, 2013). As demonstrated by a series of experimental investigations (Fryer, 1996;Clarke and Fryer, 1998;Gerke and Przybilla, 2016), the accuracy is improved by increasing convergence angles and the number of observations (intersecting rays) on the object points. This paper will focus only on the geometric calibration of a MicaSense RedEdge-M camera. Nonetheless, there are two other equally important topics related to the calibration that must be addressed when considering using a multispectral sensor. Firstly, in terms of radiometric correction, it must be remembered that spectral signatures are highly dependent on light and atmospheric conditions. This is especially important when collecting data for multitemporal analyses. Research on procedures for image acquisition in a controlled environment is still ongoing, as well as on techniques to reduce noise and methods for post-processing the images (Lucieer et al., 2014). Calibration panels and Empirical Line Methods (ELM) are widely used (Baugh and Groeneveld, 2008;Wang and Myint, 2015). Ren et al. (2013) presented a technique for spectral recalibration of a four-channel camera -Red, Green and Blue (RGB) and Near Infrared (NIR)using man-made ground targets. MicaSense proposes a method for conversion of the raw pixel values into absolute spectral radiance values using the onboard Downwelling Light Sensor (DLS) which provides irradiance values for each band (MicaSense, 2017). Mamaghani et al. (2018) proposed a new technique making use of the supplied DLS sensor of the MicaSense RedEdge camera, and an improved laboratory method (Mamaghani et al., 2018;Mamaghani and Salvaggio, 2019).
Another important topic is band co-registration, vital when using multi-band cameras. Significant misregistration errors can be experienced when relying on some software, therefore the last decade has witnessed a huge growth in research on this topic (Laliberte et al., 2011;Kelcey and Lucieer, 2012;Berveglieri et al., 2019;Shahbazi and Cortes, 2019). Jhan et al. (2018) proposed a co-registration method using SURF descriptors taking into account the differences in lens distortion and the different positions of the lenses in the camera, whereas some other authors proposed the use of SIFT descriptors (Saleem and Sablatnig, 2014;Ma et al., 2017).

Aims
The MicaSense RedEdge-M multi-lens multispectral camera (figure 1) is an interesting camera as it has some special design features with five similar lens cones and sensors. The aim of this research is to investigate these special features on the geometric calibration of each cone.
Research questions: 1. The expectation would be that the manufacture of the lenses would be from the same process and, therefore, exhibit very similar calibration parameters. However, the relationship between the lens and the imaging sensor requires a physical alignment. This could affect the location of the principal point in each case. 2. The different spectral bands may record different object points (reflections) and therefore affect the distribution of the imaged points over the format of the sensor. This could have an effect on the radial distortion and quality. 3. As this calibration process is potentially a repeatable procedure, it is important to undertake an assessment of the methodology of camera calibration approach.

Methodology
For the calibration of a multispectral sensor, a procedure similar to that applied for the recovery of the parameters of a conventional photogrammetric camera using a laboratory calibration frame can be followed. The methodology is based on the following stages: 1. The design and collection of images of the calibration frame. 2. Undertake a series of calibrations using a selfcalibrating bundle adjustment model (using Australis software). 3. Analysis of the calibration results with respect to the research questions. The primary approach will be through graphical plotting of the variations in the main camera calibration parameters.

System Hardware
The images were taken using a MicaSense RedEdge-M multilens multispectral camera (figure 1) with the specifications given in table 1. This camera simultaneously captures five bands from independent lens cones, three in the visible part of the spectrum (Red, Green and Blue) and two in the invisible part (RedEdge and Near Infrared).

Camera Calibration
An existing calibration frame within the Nottingham Geospatial Institute was used (figure 2). It boasts 85 reflective coded targets, each with a unique pattern, plus around 100 single reflective spot points. Two Interfit Tungsten 3200 studio lighting kits were used to illuminate the retro reflective targets on the calibration frame.
One was equipped with a 500W photographic light bulb with a colour temperature of 3200K. The other one was fitted with an infrared bulb to make the targets visible to the invisible bands as well. Australis v8.33 software (Photometrix, 2016) was used to undertake the automatic target recognition and generate camera calibration parameters and quality statistics. This software is a widely adopted camera calibration software, having demonstrated it provides high-quality results (Remondino and Fraser, 2006).

Test Procedure
The experimental procedure for the calibration was as follows.
The calibration frame was imaged from 21 stations using a tripod and remotely controlling the camera from a smartphone which aided stability. From the same location, images were taken with different camera rotations in order to decouple the interior (IO) and exterior (EO) orientation parameters. Also, images were taken at three distances from the frame to further strengthen the geometry (Cramer et al., 2017).
For each lens cone the calibration (using Australis software) was undertaken twice; once considering the two decentring distortion parameters (P1 and P2) and once without them. This enabled a more detailed analysis of the geometry within each cone.

Results, Analysis and Discussion
The first research question; to investigate the magnitude of the variation in the camera calibration parameters. As the lens cones are all similar, it might be expected that the parameters, for the different bands of the MicaSense RedEdge-M camera, would show similar values.      software during the self-calibration. The curves are extrapolated to the maximum radial distance (3 mm) available for each sensor (Photometrix, 2016). The five radial distortion plots generally show the same trend. However, analysing the radial distortion values at 3 mm, it can be seen that the Green and RedEdge bands are the two that most differ (the difference reaches about 9.2 μm at the sensor edges). This is almost 3 times bigger than that between the Red and NIR bands (52.6 μm and 53.9 μm respectively). The maximum radial distortion is confirmed by similar studies on conventional (RGB) consumer-grade cameras (Fraser and Shortis, 1990;Remondino and Fraser, 2006;Moe et al., 2010;Sun et al., 2017). Finally, the correlation matrices should be analysed. As an example, the interior orientation correlation coefficients for the Blue band have been reported in table 4.
To describe the radial distortion corrections, only K1, K2 and K3 have been included, whereas the two parameters K4 and K5 have been fixed to zero as they are less influential (Fraser, 1997). As expected, there is a very high correlation between the Ki coefficients (underlined in table 4). However, as reported by Cronk et al. (2006), the projective coupling between these, the other IO parameters and the EO parameters is usually low.
The two decentring distortion parameters P1 and P2 describe the result of lens elements not being centred along the optical axis. The software manual (Photometrix, 2016) suggests these parameters can be removed, as their magnitude is generally small. However, although P1 and P2 are normally not taken into account in a calibration, as the MicaSense RedEdge-M is a nonmetric camera and having five cones very close together, there is some concern that the two decentring distortion parameters might be effective and influential in the calibration.
In the first calibration, P1 and P2 were not constrained to zero and, albeit small, some values were obtained from the calibration (table 2). Then, in order to assess whether P1 and P2 are influential in providing the best quality results, they were constrained to zero. The result showed that the decentring distortion parameters had little effect on the radial distortions (figure 5), while their impact on the position of the principal point is more noticeable ( figure 4). Indeed, removing the two parameters caused a considerable shift of the principal point (on average 7.5 μm in the x-direction and 1.3 μm in the y-direction). This correlation between the decentring distortion parameters and the principal point offsets is well known, and it is further proved by the correlation matrices (circled in table 4). For the five bands, the correlations are quite strong, as they are in all cases around 0.9. It is important to highlight such a correlation, since it means that a shift in the principal point can, to a certain extent, compensate for the decentring distortion. The correlation between the decentring distortion parameters and the principal point offsets confirms what was stated by Barazzetti et al. (2011), who noticed that "there is a projective coupling between P1 and P2 with xp and yp". The obtained figures are also confirmed by Fraser's experience, who states that "correlation coefficient values of up to 0.98 are frequently encountered" (Fraser, 1997).
Although the adopted approach may be theoretically correct, it might not necessarily be significant in practical terms, especially when it comes to using some standard aerial survey software which are not always able to accommodate these two parameters.  With reference to the second research question, the distribution of the imaged points over the format of the sensor for each calibration was studied (figure 6). As mentioned in paragraph 2.2, two lamps equipped with bulbs of different wavelengths were used. This method proved to be key in making the targets visible to all the five bands of the MicaSense RedEdge-M camera as, from early trials carried out using the normal photographic light alone, the three red bands were not able to pick up as many target points.
With reference to the results in figure 5 and figure 6, while it is true that the radial distortions of the five bands increase as the distance increases, there is no clear correlation between the radial distortion and the distribution of the imaged points. Whilst it can be seen that there is a noticeable reduction of points in the xdirection at the edges of the format (except for the Blue band) there is still a reasonable number particularly in the expected useable area. The Blue and Green bands which have a more uniform distribution, even though they display a smaller number of detected points. The reds, with a recorded number of points three times higher than the Blue and Green bands, show a distribution which is more clustered around the centre. This different density could be due to both the nature of the two bulbs used, as their wattage is different, and the fact that the majority of coded targets were not picked up as such but as single reflection points when performing the calibration of the Red, NIR and RedEdge bands. Having said that, the illumination approach appears to work well with the employed photogrammetric network. In order to improve the calibration (research question 3), further studies involving the illumination, perhaps making use of different light settings with bulbs of different wavelengths could be undertaken.

CONCLUSIONS
Camera calibration parameters for each individual band have been presented and compared, and plots of the radial distortion profiles and positions of the principal point for each band have been included.
The first research question has shown that the lenses have different but very similar radial distortion characteristic. The positions of the principal points vary significantly between the cones. The lens decentring values P1 and P2 can have a significant effect on the position of the principal point.
The second research question has shown that a high number of object/target points can be recorded across all bands by using a combination of a photographic and infrared light.
The third research question; further work on illumination of the targets, particularly for illumination of the red band sensors may provide a more even coverage of the sensor format.

RedEdge band
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition)