TWO METHODS FOR SELF CALIBRATION OF DIGITAL CAMERA

Photogrammetric mapping using Commercial of the Shelf (COTS) cameras is becoming more popular. Their popularity is augmented by the increasing use of Unmanned Aerial Vehicles (UAV) as a platform for mapping. The mapping precision of these methods can be increased by using a calibrated camera. The USGS/EROS has developed an inexpensive, easy to use method, particularly for calibrating short focal length cameras. The method builds on a self-calibration procedure developed for the USGS EROS Data Center by Pictometry (and augmented by Dr. C.S Fraser), that uses a series of coded targets. These coded targets form different patterns that are imaged from nine different locations with differing camera orientations. A free network solution using collinearity equations is used to determine the calibration parameters. For the smaller focal length COTS cameras, the USGS has developed a procedure that uses a small prototype box that contains these coded targets. The design of the box is discussed, along with best practices for calibration procedure. Results of calibration parameters obtained using the box are compared with the parameters obtained using more established standard procedures. Work performed under U.S. Geological Survey contract 08HQCN0005


INTRODUCTION 1.1 General
For any photogrammetric project, an accurate knowledge of the sensor/camera's interior orientation parameters is necessary.In this research, we shall present two methods used by the USGS to determine these parameters for small and medium format digital cameras.The first method, developed by Pictometry (augmented by Dr C.S. Fraser), uses a series of coded targets on a cage.The coded targets are placed on the cage in three different planes, which allows for a robust calibration procedure.The second method describes the development of a method whereby the coded targets are pasted on a small prototype box.The importance of calibrating a camera used for photogrammetric purposes cannot be overstated.The interior orientation parameters of a camera help in determining the exact path of a ray of light that enters a camera, at the time of exposure.The main interior orientation parameters are the focal length of the lens and the location of the principal point of symmetry.The knowledge of the deviation of the light ray from a straight line, described by polynomial coefficients, is also important.This deviation is termed lens distortion, and the polynomial coefficients are termed lens distortion parameters.Photogrammetric projects can be executed without a thorough knowledge of the calibration parameters too.However, these would require a very dense network of control points that will render these projects prohibitively expensive.

Camera calibration methods
Camera calibration methods preferred by photogrammetrists can be categorized broadly into three classes, In-situ calibration, calibration using precise multi-collimator instruments and selfcalibration.

In-situ calibration:
In-situ methods require In-situ calibration methods require an area (a calibration range) with a very dense distribution of highly accurate control points.The control points in the calibration range should be well distributed in the horizontal, as well as in the vertical direction.The in-situ method requires aerial imagery over a calibration range.A rigorous least squares block adjustment based on the colinearity equations, augmented by equations modelling radial and decentring distortion (Eq.5) can generate accurate calibration parameters.On many occasions, these calibration procedures are used to validate photogrammetric data (Cramer and Haala, 2009), or validate/augment the calibration parameters determined in the laboratory.The calibration range should be maintained periodically, and may include re-survey of the control points, making sure they are undisturbed etc.

Precision multi-collimator instruments:
The USGS operates a multi-collimator calibration instrument located at Reston, Virginia, USA (Light, 1992).The instrument is used to calibrate film based cameras, and while digital cameras are increasingly used, there are a number of photogrammetric companies that still employ film cameras.The aerial camera is placed on top of the collimator bank, aligned and focused at infinity.Images that capture the precision targets located in telescopes lens (of the multi-collimator) are taken.The deviation of the measured image (x, y) coordinates from the known (X, Y) coordinates forms the basis for solving for the calibration parameters (Eq.5).

Self calibration:
Self calibration uses the information present in images taken from an un-calibrated camera to determine its calibration parameters (Fraser, 1997;Fraser 2001;Remendino and Fraser, 2006;Strum, 1998).Methods of self calibration include generating Kruppa equations (Faugeras et. al., 1992), enforcing linear constraints on calibration matrix (Hartley, 1994), a method that determines the absolute quadric, which is the image of the cone at a plane at infinity (Triggs.While there are many techniques employed by researchers (Hartley, 1994;Faugeras et al., 1992), most of these do not find solutions for distortion and principal point, as they are not considered critical for Computer Vision.On the other hand, for photogrammetrists, these are critical parameters necessary to produce an accurate product at a reasonable price.
In this research, self-calibration techniques are used to determine camera calibration parameters.Section 2 provides a brief theoretical framework for calibration.It goes on to discuss the design of two methods for self calibration used at the USGS, and describes the experimental set-up.It introduces an inexpensive method for calibrating small and medium format digital cameras, with short focal length.Section 3 analyses the results of calibration, and compares the results obtained from the two methods described in Section 2. Section 4 presents the conclusions and discusses future work.

Theoretical basis
The self calibration procedure described in this research is based on the least squares solution to the photogrammetric resection problem.The well known projective collinearity equations form the basis for the mathematical model.is the camera orientation matrix.Since the lens in the camera is a complex system consisting of a series of lenses, the path of light is not always rectilinear.The result is that a straight line in object space is not imaged as one in the image.The effect is termed distortion.Primarily, we are interested in characterizing the radial distortion and de-centring distortion.Radial distortion displaces the image points along the radial direction from the principal point (Mugnier et al., 2004).The distortion is also symmetric around the principal point.The distortion is defined by a polynomial (Brown, 1966;Light, 1992).
The (x,y) components of the radial distortion are given by: (3) The second type of distortion is the decentring distortion.This is due to the displacement of the principle point from the centre of the lens system.The distortion has both radial and tangential components, and is asymmetric with respect to the principal point (Mugnier et al., 2004).The components of de-centring distortion, in the x-y direction are given by A third distortion element, specific to digital cameras accounting for scale distortion of pixel sizes in the x and y direction is also incorporated The final mathematical model is a result of adding Eqs. 3 and 4 and 5 to the right hand side of Eq.The camera calibration facility is located at the USGS's Earth Resources Observation and Science (EROS) Data Center in Sioux Falls, South Dakota.Fig. 1 shows the position of the calibration cage, with respect to the room.Also shown are some of the positions for locating the cameras.The cage consists of three parallel panels.Each panel has a number of circular retroreflective targets (dots), and a few coded targets (Fig 2a).The coded targets are so referred because the pattern of the placement of the individual circular dots that make up these targets is unique (Fig. 2b).Each coded target has five dots that are positioned in the same relative orientation as the red lines shown in Fig. 2(b).The intersection of the red lines is taken as the centre of the coded target.
For the calibration procedure, the camera lens is always focussed at infinity (unless the camera is used for close range purpose, in which case the focus is fixed at the specified distance).The choice of the distance of the camera from the front panel of the cage depends on the focal length of the camera, and the depth of focus that has been selected.Once the camera-cage distance is fixed, three angular positions from the centre of the front panel of the cage are selected, keeping in mind the optimal angles for convergent photography, and the limitations imposed by the dimensions of the calibration room.Ideally, the angular positions will be close to what is shown in Fig 1 .Once the images are captured, they are processed using software called Australis (Fraser, 2001).Australis uses a free network method of bundle adjustment.It recognizes the patterns in the coded targets and calculates their centre.Since it is a free network solution, the least squares iteration converges easily, and a relative measure of the geometry of the system (the lens, camera, and the targets) is obtained.

Camera self calibration using a box
With the ever increasing use of small platform based digital photogrammetry (such as UAV, etc.) for aerial mapping, the USGS developed a self-calibration procedure for small format cameras that does not require establishing a large calibration cage.Instead, a smaller rigid box that can be easily designed and constructed is used.The current design of the box is as shown in Fig. 4(a).The box is designed such that its dimensions are approximately 24 inches at the top (outer edge) and 12 inches at the bottom (inner).The inner walls of the box are not vertical, but are sloping at approximately 30 degrees.A scaled down series of coded targets are pasted on all the interior surfaces of the box.The design takes advantage of the simplicity of the free network bundle adjustment solution that requires no outside control structure.) with an increased number of coded targets and circular targets was also designed for use with cameras with a very small field of view.For calibration photography, the optic axis of the camera is usually kept parallel to the inclined interior walls of the box.Three images are obtained from each side, and one image is obtained from each of the four corners, which results in a total of sixteen images.The images are alternatively taken in portrait and landscape modes.However, for cameras to be used with the smaller box, the photography is along parallel axis, mimicking the collection of vertical aerial images for a photogrammetric project.For a stable solution, as many targets as possible are obtained from the corners of the camera lens.

RESULTS AND ANALYSIS
Two cameras are analysed in this paper.The first one is a Nikon D1x digital single lens reflex (SLR) camera with a 20mm focal length lens (Nikkon AF) was used for this research.F # of 8 was chosen for calibrating with the cage as target, and f # of 22 was chosen for calibrating with the box as the target.The second camera is also a Nikon D70 camera with a 70mm-180mm lens.This camera was used to make photogrammetric measurements on the Fragment C of the Antikythera mechanism (Evans et. al., 2010).The camera's (hence called Puget Sound camera) focal length was fixed at around 70mm by the researchers working to measure the Antikythera.However, to determine the exact focal length and the geometric distortions due to the lens, the Puget Sound camera and lens were sent to the USGS EROS.

Results
The results of the experiments are shown in Fig2.6-8 and Tables 2-3.A total of 15 images were of the cage taken for use in the calibration software.Usually, only nine images are used.However, in this case 15 images were required to completely cover all the targets.The free network bundle adjustment solution is graphically displayed in Fig 7 (a).In a similar manner, the hyperfocal distance for the calibration using the box was calculated at 1 ft.A total of 20 images were obtained for the box.The free network solution is graphically shown in  Table 2 shows the solutions to the bundle adjustment and the calibration parameters obtained from the two experiments.Table 2 lists the calibration parameters that were obtained as a part of the bundle adjustment solution.Fig. 7 shows the plots of radial distortion obtained while using the cage (Fig. 7a) and while using the box (Fig. 7b)

Analysis
The results from the bundle adjustment are initially calculated in the camera pixel space.The results are converted into real world coordinates by using the pixel size as the scale factor.This replaces the fiducial marks used in film cameras.Therefore, each pixel (or the average pixel) is considered to the equivalent of the measurements from the fiducial mark.The results of the two calibration procedures indicate that the parameters are close to being identical (Table 2).The charts in Fig. 7a and 7b also show the similar results.However, in our experiments, we found that the results start varying if the camera is positioned too close to the targets.This observation seems consistent with previously reported studies on close range photogrammetric camera calibration (Brown, 1971).However, more analysis needs to be done for anything conclusive.Table 3 lists the results for the results from the Puget Sound camera.The field of view of the Puget Sound camera was too small to be used with the cage or the box described above.Hence, an even smaller box with a more dense distribution of coded and circular targets was constructed.The results indicate a large principal point offset, as well as a higher than usual deviation from the nominal focal length of 70mm.
Since the box as a calibration target is meant for small format short focal length cameras, the distance between the targets and the cameras should be close enough so that the software is able to recognize the targets.The size of the targets, therefore, needs to be selected accordingly.

CONCLUSIONS
In this research, two methods of camera calibration that are used at the USGS EROS at Sioux Falls, South Dakota, USA were presented.The camera calibration lab is housed primarily to calibrate medium format digital cameras, with a focal length range between 20-120mm.The main calibration method uses the principles of self calibration and bundle adjustment on coded targets located on an aluminium cage.A second method to perform calibration was presented.This method used a scaled down version of the coded targets pasted on a small rigid box.
Both the methods involve taking images of the targets from different camera locations and orientations.It was shown that the solutions camera calibration parameters obtained from both the methods are close to each other.The same time the approach using the box yields promising results and can be used for verification of the calibration parameters.
Eq. 1, (x ,y) are the measured image coordinates of a feature and ( , x p p y ) are the location of the principle point of the lens, in the image coordinate system, f' refers to the focal length and

Figure 1 .
Figure 1.Layout of the calibration lab and the calibration cage Figure 2. (a) Image of the calibration cage, with three panels (b) the pattern in a coded target and (c) the individual circular target Figure 4(a).A rigid box design for calibration of small format cameras and (b) another design with an increased number of targets A smaller box (Fig. 4(b)) with an increased number of coded targets and circular targets was also designed for use with cameras with a very small field of view.For calibration photography, the optic axis of the camera is usually kept parallel to the inclined interior walls of the box.Three images are obtained from each side, and one image is obtained from each of the four corners, which results in a total of sixteen images.The images are alternatively taken in portrait and landscape modes.However, for cameras to be used with the smaller box, the photography is along parallel axis, mimicking the collection of vertical aerial images for a photogrammetric project.For a stable solution, as many targets as possible are obtained from the corners of the camera lens.

Figure 7 Figure 6 .
Figure 6.Graphical representation of the bundle adjustment solution for (a) Cage and (b) Box based camera calibration

Figure 7 .Figure 8 .
Figure 7. Radial distortion plots showing the distortion (Y-axis, µm) as a function of distance (X-axis, mm) from the principal point for results of camera calibration obtained from (a) Cage and (b) Box.The plots are obtained from Australis software

Table 1 .
List of interior orientation parameters

Table 3
Camera calibration parameters for the Puget Sound Camera