BIFOCAL PAIRS OF IMAGES FOR LOW-COST SURVEY IN CLOSE RANGE PHOTOGRAMMETRY

Abstract. This paper present a method for close range photogrammetry based on an camera positioning scheme in which two cameras capture an equal portion of an object at the same scale, but have different focal lengths and camera-to-object distances. This scheme is alternative to the stereoscopic scheme and is associated with a system of equations which permits one to calculate first the relief displacement of points on a photograph and then their relief relative to a reference plane. The obtained relief and relief displacement values can be used to produce low-cost orthophotographs by using software for image processing, which doesn’t need to be dedicated, but has to provide measurement and calculation functions. Moreover, this method also allows one to obtain three-dimensional coordinates, through further calculations.



INTRODUCTION
In a perspective view, relief displacement is a shift that occurs when a point placed at some distance from a reference plane is projected by an inclined ray. It generally constitutes a disturbing factor for the geometry of an image. In the described method, however, it is employed as a useful element. In order to make this possible, it was firstly necessary to address the issue of how to determine relief displacement values in a simple way. Generally, algorithms for calculating relief displacement require the knowledge of the relief of a point; in turn, to know the relief of a point, a photogrammetric or a topographic survey is required. This means that it is necessary to examine the morphology of an object, before relief displacement can be used for surveying surfaces. The described method overcomes the above problem by using a system of equations and a camera positioning scheme which permits the calculation of the relief displacement of a point without previously knowing its relief. Based on this information, it is possible to determine the relief of a point relative to a reference plane and subsequently calculate threedimensional coordinates and produce orthophotographs, through simple calculations and non-dedicated software. This can be an effective alternative to the stereoscopic scheme when performing a low-cost survey in close-range photogrammetry, especially if usual procedures are not applicable. In the paper, the method will be described as follows: 1. Traditional camera positioning schemes 2. Alternative camera positioning scheme 3. Traditional calculation of relief displacement 4. Alternative calculation of relief displacement and relief 5. Some consideration about the new method 6. Calculation of coordinates

Traditional camera positioning schemes
In close-range photogrammetry, photographs are generally taken according to stereoscopic camera positioning schemes, in which images are taken from two different points of view (Kraus, 1998) and (Selvini, 1998). The result is a pair of images, each representing, for a certain percentage of its surface, an object or a portion of an object, from two different viewpoints ( Figure 1).

Figure 1. Stereoscopic scheme (planimetric projection)
The stereoscopic scheme allows the formation and exploration of a stereoscopic model which, when the restitution is carried out by an operator, facilitates the identification of homologous points. However, today's digital software offers more flexible schemes of camera stations compared to past applications. Based on the potentiality of current digital applications, some new methods have been developed and are described below. One of the new methods for image acquisition is called Structure-from-Motion (SfM). It is based on a camera moved in space and taking images; so, overlapping images can be acquired from multiple viewpoints. The particular camera pose and scene geometry are determined simultaneously and automatically, using a highly redundant bundle adjustment based on matching features in multiple overlapping, offset images (Westoby et al., 2012). Scale can be set using nonspecialist technology (e.g. standard laser rangefinders). However, the system is generally equipped with other sensors, such as RTK-GNSS (Real Time Kinematic-Global Navigation Satellite System) and IMUs (Inertial Measurement Units), for directly georeferencing and scaling the survey (Smith, 2016). Alternatively, GCPs (Ground Control Points) with XYZ coordinates can be used (Micheletti, 2015) and (Smith, 2016). The SfM does not require expensive equipment or specialist expertise, and under certain conditions can produce point clouds with quality comparable to existing survey methods, such as Terrestrial Laser Scanning (Smith et al., 2016). Another solution is the three-line scanner scheme, particularly suitable for digital applications in aerial photogrammetry ( Figure 2). In the three-line scanner scheme, an object is photographed from a view point with forward, nadir and backward-looking linear arrays, to provide three possible stereoscopic pairs of images that are forward-to-backward, forward-to-nadir and backward-to-nadir, (Leica, 2015) and (Zang, 2004). It is also worth mentioning the older Rolleimetric system of Galileo, in which an object is photographed from three points of view to provide three successive frames and the three corresponding points are collimated without stereoscopic observation (Figure 3), (Fangi, 1988). This system can be developed in analogical or digital form. Other schemes are applied in artificial intelligence, based on two images taken from a camera undergoing a known translation across space. A special case is when the translation is made along an optical axis. This scheme yields two images at a different scale. The same result is obtained by the "zooming to depth" method (May, Olsen, 1990), in which two images are taken by a camera from a single point of view at a different scale. In any case, the image displacement can be estimated by using a gradient-based scheme, or a feature-based scheme or a line-based scheme. (May, Olsen, 1990). The method described in this paper uses images taken from two different points of view positioned along the optical axis; unlike the previous scheme, it permits one to obtain two images at a same scale, so that the same portion of scene is framed by the bifocal pair. In these images relief displacement is used for carrying out photogrammetric calculations.

Alternative camera positioning scheme
The described scheme of camera stations ( Figure 4) enables one to obtain two images that frame the same portion of an object at the same average scale. The two points of view are preferably positioned in the same direction perpendicular to the object, that is: the second point of view is positioned along the optical axis of the first camera (Midulla, 2015). To obtain a corresponding scale between the two images, the photographs are taken by imposing between the two camera-to-object distances d1 and d2 the relationship that exists between the principal distances c1 and c2 of camera 1 and 2 respectively. The planes of the images are preferably set parallel to the plane of photographed surface, whereas the centres of the images capture the same object point, to ensure that the origins of the image coordinates coincide. We will call this kind of photographs "bifocals". Apart from the possible deformation due to the distortion of the lenses, in such images the corresponding points which lay on the reference plane have the same image coordinates, whereas the corresponding points which protrude or recede in relation to the reference plane have, between them, different image coordinates. This is due to the relief displacement that will be described below.

Traditional calculation of relief displacement
As said before, points protruding or recessing from a reference plane present a shift in a photograph called relief displacement ( Figure 5). Relief displacement is generally calculated with the following formula, (Kraus, 1998) and (Selvini, 1998): In which: e = relief displacement on the image s = distance from the nadir point to the image point P' r = protrusion or recess of a point P from the reference plane d = distance from the perspective centre to the reference plane The value e is referred to the radial relief displacement when the value of s is referred to the radial distance from the nadir point to the displaced point. From the above formula, it follows that the relief displacement, in absence of the value r, is not calculable. Now we can calculate the r value with the same formula (1), rewritten as follows: However, from this formula (2) it follows that r can be calculated only if the e value is known. Based on this consideration, an algorithm was conceived to solve both the problem of the determination of the relief displacement and the consequential calculation of r, for which the bifocal scheme constitutes an indispensable prerequisite.

Alternative calculations of relief displacement and relief
Starting from the assumption that the corresponding points within a pair of bifocal images have a different relief displacement when they have a different relief (Figure 6), it is possible to write two equations of type (1), referred to each of the two images 1 and 2: or, of type (2): In which: e1= relief displacement on image 1; e2 = relief displacement on image 2; s1 = distance of a point from the nadir point on image 1; s2 = distance of a point from the nadir point on image 2; d1= camera 1-to-reference plane; d2 = camera 2-to-reference plane.
Generally, in the restitution phase, d2 corresponds to the distance at which the reference plan is positioned; d1 is preferably calculated as d2+Δd, where Δd is measured in site; s1, s2 are measurable on the image; h, e1 and e2 are unknown. Measurements on the image are carried out with reference to a system of Cartesian axis with its origin on the centre of the image, with axis x and axis y respectively parallel to the two main dimensions of the image.
The resulting system (two equations and three unknowns) therefore is not solvable. However, it is possible to rewrite the two equations, replacing e2 with the e1 + Δs, in which Δs = s2 -s1 ( Figure 6). Figure 6. Relief displacement ne1 and ne2 on the reference plane Therefore the system becomes: or: These systems are now composed of two equations with two unknowns (r and e1), and are therefore solvable. By developing the above systems of equations it is possible to determine the relief displacement on the two images, as well as the relief of the points in relation to a reference plane.
The effectiveness of these calculations was verified through a case study (Midulla, 2015) carried out with the use of a digital photographic camera Canon D300 equipped with a CMOS sensor of a size 22.7 x 15.1 mm, resolution of 3072 x 2048 pixels, and a total of 6.3 effective million pixels. It is a nonmetric camera; therefore it will be necessary to take this into account when examining the obtained results. Focal lengths of 19 mm, 28 mm and 35 mm (equivalent to an analogical format of 28 mm, 45 mm and 50 mm) were used.The images were taken from distances of about 9,5 m, 14 m and 17,5 m, according to the scheme of the camera stations illustrated in Figure 4. During the shot, the camera was placed on a tripod, with the image plane parallel to the wall, centred on the centre of the wall for all the photos. Three dimensional targets were used.The space resolution of the images is equal to 3 mm, the scale is 1:500. Two different pairs of images were obtained. One pair consists of images taken with a 35 mm and 19 mm focal length; the other one consists of images taken with a focal length of 35 mm and 28 mm. In total 23 points have been collimated which reproduce relief of various sizes between 1 cm and 100 cm. The calculations have been carried out onto the spreadsheet software Microsoft Excel, on an electronic sheet prepared for the automatic calculation of the relief displacement and the height. To verify the reliability of the results, the value of the relief, obtained with the above formulas, were then compared with the values measured on site, calculating the errors and the RMSEs (Root Mean Square Errors) relative to the two groups of points: that of the images taken with 35 and 19 mm focal length, and that of the images taken with 35 -28 mm focal length. The accuracy obtained is equivalent to 2,6 cm RMS value with a maximum error of 4.9 cm, for the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W17, 2019 6th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 2-3 December 2019, Strasbourg, France configuration 35 -19 mm; equal to 3,4 cm RMS, with a maximum error of 6,1 cm, for the configuration 35 -28 mm.

Some considerations
The results of the operations carried out have demonstrated that it is possible to utilise bifocal images for the restitution of the relief of points in relation to a reference plane and and that it is preferable to use pairs of images taken with lenses with a strong difference of focal length. Moreover, it is possible to consider the following limits: (a) At the nadir point, there is no relief displacement, whereas, in an area around the nadir point, the relief displacement can be difficult to determine, due to the modest amount of the shift of the points, above all in the image taken with the longest focal length camera. The amplitude of the area in which the shift of the points is modest depends on the focal length, on the relief of the point, and on the resolution of the image. We will call that area 'area of uncertainty'. From this follows that each time, based on the morphology of the object, the focal length, scale and resolution of the camera must be carefully chosen, so as to limit the area of uncertainty. All other parameters being equal, the more uneven are the surfaces at the centre of the image, the smaller the scale of the photograph can be. However, it is important to consider that this limitation can be overcome in the production of orthophotographs. This is true as the area of uncertainty occurs only when the relief displacement is so small that can not be measured. Therefore, in order to produce an ortophoto, no correction in the area of uncertainty is required, especially when the bifocal image with a greater focal length is the one that is ortho-projected.
(b) The influence of the error of the measurement of the radial distances s1 and s2 is direct but contained within the calculation of the relief displacement, indirect but amplified by the ratio d/s within the calculation of r. As a consequence, the influence of the measurement error is more significant, above all, near by the nadir point. With regard to point (a) and (b), the results are more reliable for larger scales as well as for high resolution images.
(c) The use of two different lenses within the pair of bifocal images involves a different distribution of the distortion on the two images, and therefore it is appropriate to carry out a pretreatment which corrects the images. The use of a metric camera guarantees that the lens distortion is known. In the case of a non-metric camera, these data are obtained with selfcalibration.
(d) The absence of a stereoscopic model can make recognition of the corresponding points difficult. As a consequence, it would be appropriate to apply this method in the case of objects with a well-defined geometry, if targets are to be avoided. The described method is suitable, therefore, in the field of close range photogrammetry, for the restitution of the façades. The recognition of the corresponding points can however be facilitated by the visualization of the images on two superimposed layers, of which the opacity can be changed to make simultaneously visible the two images. It can also be facilitated by collimating corresponding points along the radial directions. However, this limitation do not occur in automatic applications of the described method.

Calculation of coordinates
Having calculated the relief displacement e and the relief r, it is possible to calculate the absolute coordinates of a point in a expeditious manner, according to the procedure described below.
The radial distance s measured on one of the two bifocal images is corrected by removing the relief displacement calculated with formula (5), obtaining s' (fig. 7). Figure 7. Radial distance s1, relief displacement e, angle θ and corrected radial distance s'1 s' is subsequently divided into the two components s1'x and s1'y: in which the angle θ can be measured on the image together with the radial distance s. For example, the above formula can be applied for the production of orthophotographs and for quick restitution. Therefore, as a result, for each point coordinates in a local reference system can be obtained, which can be transformed into coordinates within an absolute reference system. This can be achieved with a transformation which can be, for example, a roto-translation with variation of scale, by using reference points, that is points whose coordinates are known in the two reference systems.

CONCLUSIONS
In conclusion, the proposed method allows us to survey the morphology of a surface, as well as to achieve orthorectification and expeditious restitutions, through simple calculations and non-dedicated software. The following advantages are worth mentioning: -100% overlap between photographs; -a rapid calculation of relief relative to a reference plane; -the use of general software; -the possibility to use a camera positioning scheme which may be an alternative when the conformation of the places doesn't allow the operator to position the camera according to the stereoscopic scheme. However, this procedure presents some limitations as shown below: -a rigorous camera positioning scheme; -the necessity to know two lens distortion curves; -the difficulty of determining the relief displacement around the nadiral point; -the difficulty of recognising the corresponding points; -the influence of the error of measurement, particularly insidious near the nadiral point. The latter two limitations do not occur in the production of orthophotographs and automatic restitution. For all these reasons, the above photogrammetric method can represent an effective alternative for the purpose of surveying terrain, buildings and objects, especially when usual procedures are not applicable and orthographs are produced. s 1 'x = s 1 ' sen θ ; s 1 'y: = s 1 ' cos θ (7)