CALIBRATION OF PHOTOGRAMMETRIC SYSTEM FOR 3D MEASUREMENTS IN INACCESSIBLE MULTIMEDIA WORKING SPACE

Hydrodynamic tunnel has proved to be an effective mean for studying aerodynamic processes using scaled models of real aircrafts. The specificity of study in a hydrodynamic tunnel requires to apply non-contact measuring techniques, such as photogrammetry, for retrieving information on geometrical characteristics of the flow. So reliable calibration of photogrammetric system is a key element for trustworthy analysis of flow behaviour. For 3D measurements in a hydrodynamic tunnel calibration technique must account for refraction effects at optical media interfaces. But often the design of a hydrodynamic tunnel does not allow to perform standard calibration procedure based on image acquisition of a special test field, placed in the working space of the measuring system. The presented study addresses this problem – developing the technique for accurate photogrammetric calibration, for the case of the working space being inaccessible for placing there a test field. The developed calibration technique estimates parameters of interior orientation of the photogrammetric system and parameters of multimedia optical environment by special preliminary procedure, that allows to obtain accurate 3D measurements during experiments in hydrodynamic tunnel. Experimental evaluation of the developed technique demonstrated high accuracy of photogrammetric 3D measurements.


INTRODUCTION
The study of flow process over aircraft surfaces is very important part of the design and exploitation the aircraft, that has to cover various flight modes, including critical conditions of operating such as icing or critical angles of attack. Real flight experiments in these conditions are strongly unrecommended or just impossible, so modelling in wind or water tunnel became the powerful and flexible tool for research in aerodynamics and hydrodynamics. The modelling is performed using scaled models of a real aircraft or its part depending on the problem to be investigated.
The correctness of the results obtained by the modelling is guaranteed by high level of similarity of real process and the modelled one, that is based on the theory of similitude. According to similitude concept a model process corresponds to considered real process in case if the former is geometrically, kinematically and dynamically similar to the latter. Such similarity is provided is if both processes have equivalent values of some dimensionless quantities describing the process characteristics. For aerodynamics these dimensionless parameters are Reynolds number (characteristic of viscosity), Mach number (characteristic of speed of sound), Prandtl number (characteristic of thermal conductivity), and their equivalence to real process indicates to the validity of experiment results.
Similitude concept allows to test aerodynamic flow process in a hydrodynamic tunnel at significantly lower velocities of the flow then in real flight condition. This option along with advanced possibilities of flow visualization makes a hydro- * Corresponding author dynamic tunnel a convenient tool for flow study in complicated cases such as aircraft icing.
Flow visualization, as a simple but effective technique for its behaviour study, is widely used in aerodynamic research. Usually small particles or colour inks serve for flow visualization ( Figure 1). Such approach gives useful information about flow behaviour, but do not provide with quantitative characteristics of the flow process. Therefore accurate 3D measuring is necessary and essential part of experiments in a hydrodynamic tunnel, that is needed for in-depth analysis and forecasting the evolution of critical flow processes in real flights.
Photogrammetric measuring technique seems to be the best approach for the tasks of acquiring 3D data in experiments in a hydrodynamic tunnel due to non-contact principle of measurement, significant volume of registered information and high accuracy of the method. High accuracy of the photogrammetric measurements is provided by preliminary calibration of photogrammetric system, that have to account for all important conditions of the imaging process. In case of 3D measuring in hydrodynamic tunnel, imaging model and calibration technique should, in first turn, take into consideration refraction at the interfaces of different optical medias (air, glass, water). system calibration in multi-media optical environment for the case when working space is inaccessible for placing test field there.
The main contributions of the study are: (1) the technique of photogrammetric system calibration for accurate 3D measurements in multimedia optical working space for the case when working space inaccessible for placing test field there; (2) experimental estimation of the calibration accuracy for different calibration options; (3) experimental evaluation of calibration accuracy for multi-media 3D measurements by the proposed technique in comparison with standard calibration procedure.

RELATED WORK
With undeniable advantages such as high accuracy, the possibility of non-contact measurement of dynamic processes and high productivity, photogrammetric 3D measurement technology has become in demand in a wide variety of applications. Although most part of such applications requires operating in single optical media, some special cases deal with several optical media when imaging an object. So, the problem of photogrammetric system calibration for working in optical multimedia environment attracted attention of scientific groups since the middle of the XX century (Barnes, 1952, Bass, 1966. Significant part of studies addressed to underwater measurements and underwater camera calibration. The applications aimed at underwater research are the prevailing part that need to consider refraction effects in imaging process. Among these such application as study of seabed topography along with 3D measurement of complex structures located underwater (Bianco et al., 2013, Nocerino andMenna, 2020). A set of industry task uses vision based techniques for underwater monitoring of energy production facilities (Leatherdale andTurner, 1983, Baldwin, 1984), ship hulls (O'Byrne et al., 2014) etc. Archaeological research has a need for underwater monitoring, 3D measuring and 3D reconstructing of archaeological sites (Drap et al., 2015). The comprehensive review of camera calibration techniques for accurate measurement underwater can be found in (Shortis, 2019).
Various approaches were proposed for 3D measurement in multimedia optical environment. Some of them exploits the effect of vanishing of refraction in case of incidence angle being equal to 90 degrees. To achieve this additional optical elements are included in measuring system. Applying prisms to provide incidence angle of 90 degrees is a common solution for fluid flow study by stereoscopic particle image velocimetry (Raffel et al., 2018).
Another approach to achieve high accuracy of vision-based 3D measurements is to account refraction in imaging model and to perform special calibration of the measuring system (Sedlazeck and Koch, 2012, Murase et al., 2008, González-Vera et al., 2020. Including ("absorbing") the influence of the refraction in camera calibration parameters is rather close description of distortion effect in case of camera optical axis being perpendicular the plane (or dome port) of camera. In this case general refractive effect at the media interface has radial symmetry relative to the principal point.
The described above method of "absorption of refractive effects" has such drawback as a presence of some systematic errors not accounted in the model. The refractive effect destroys the assumption that the camera has a single projection center (Sedlazeck andKoch, 2012, Chadebecq et al., 2019), being the basis for this model.
The estimation of the error introduced into optical multi-media 3D measurements with taking no account for refraction allowed to propose an implementation methodology in a modern photogrammetric workflow based on structure-from-motion and multi-view stereo techniques (Skarlatos and Agrafiotis, 2018). The proposed refraction correction approach is applied at the photo level, thus allowing to be implemented within existing technologies. Results obtained for two test sites against reference data has demonstrated that the proposed algorithm can reduce the errors caused by refraction to two times, regardless of the depth.
Alternative approaches offer solutions for geometric correction by introducing a virtual projection center (Telem and Filin, 2010) or a two-stage correction (Bräuer-Burchardt et al., 2015), including an initial standard calibration in the air, followed by the introduction of additional parameters describing the effects of refraction at the boundaries of optical media.
In underwater imagery, the image formation process includes refractions that occur when light passes from water into the camera housing, typically through a flat glass port.
The extension the existing works on physical refraction models by considering the dispersion of light allows to derive new constraints on the model parameters for use in calibration (Yau et al., 2013). The proposed novel calibration method achieves improved accuracy compared to existing work demonstrated by evaluation of the method through synthetic and real experiments.
The image correction methodology (Agrafiotis et al., 2020) for aerial image-based bathymetric mapping exploits machine learning techniques for accounting for water refraction. Machine learning is used for recovering depth from image-based dense point clouds, and further correction of refraction on the original imaging dataset. After that, the refraction-free aerial image datasets are processed by structure-from-motion and multi-view stereo techniques. The experiments performed on four image datasets demonstrated high quality of the resulting bathymetric maps.

MATERIALS AND METHODS
Accurate 3D measurements of flow during experiments in aerodynamic or hydrodynamic tunnel is an essential requirement for correct understanding of flow behaviour. Another important requirement is to perform measuring without disturbing the flow. Photogrammetric technique meets both of these. But for applying photogrammetric measurement system for experiments in a hydrodynamic tunnel it is necessary to take into account refraction at interfaces of different optical media.
Accurate calibration technique for 3D measurements for the case, when the object to be measured is located in multimedia optical environment, was developed and evaluated as a part of the research project on aircraft icing study by modelling in a hydrodynamic tunnel (Knyaz et al., 2020, Knyaz et al., 2021b. This technique was developed basing on supposition that calibration test field can be placed and manipulated in working space with different optical media (liquid). But design of a hydrodynamic tunnel often does not allow to manipulate with a test field in a working space, when it is filled with liquid. So a special technique for this case is developed.

Hydrodynamic tunnel
Hydrodynamic tunnel allows studying flow behaviour on scaled model of aircrafts in controlled conditions. Basing on similitude concept the experiments in hydrodynamic tunnel can be carried out at very low flow velocity (flow velocity V = 2 − 10cm/s) to be similar real condition of flight. Equivalence of Reynolds numbers for model process and real conditions provides the correctness of the results for an aerial vehicle model tested in water.
The experimental study of flow behaviour is to carry out in the hydrodynamic tunnel HDT-400 developed in Central Aerohydrodynamic institute (TsAGI), Russia. HDT-400 has working space of 400 × 400 × 400 mm. Flow with given speed is formed due to the gravity of the water flowing into the working part from a large container located above. At the same time, the transition from this capacity to the working part is specially organized in the form of a confuser made, and the beginning of the working part is equipped with a honeycomb.
The hydrodynamic tunnel is designed in such way, that it is not possible to place calibration test field in the working space when the working space is filled with water. The studied model of aircraft is placed in the hydrodynamic tunnel before water starts flows through the working space. Then the working part is locked hermetically, and water flows through. The duration of flow process depends on the capacity of water tank, and it is about several minutes for HDT-400. So a special calibration technique for 3D measurements in HDT-400 is required.
To perform preliminary experiments and evaluate the developed technique the laboratory setup was developed (Figure 2).
It includes a water tank and a working space where tested model can be placed for flow study. The laboratory setup allows to perform calibration and test 3D measurement in condition close to the hydrodynamic tunnel HDT-400.
The photogrammetric 3D measurement system includes two DMK 37BUX273 cameras with the IMX273LLR Sony CMOS sensor and structured light projector mounted on a rigid platform, providing stable exterior orientation.

Imaging model for two optical media interfaces
For optical 3D measurements in the hydrodynamic tunnel accurate model of imaging in case of multi-media optical environment has beed developed at the previous stages of the study (Knyaz et al., 2020). The model considers light ray path from object point A to the corresponding image point a runs through two optical media interfaces, that change the direction of the ray so that standard (one optical media) co-linearity equations becomes not valid. Figure 3 shows coordinate systems, used in the imaging model. These are: object coordinate system OXY Z, glass system of coordinates ΩXgYgZg and image system of coordinates Cxyz. The ray from the object point A to the image point a changes direction at media interfaces (liquid-glass and glass-air), and can be presented as three vectors: r 1 for air, r 2 for glass, and r 3 for liquid. For vector r 1 with image coordinates (x, y, −c) T , its coordin-The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France ates in the object coordinate system (X1, Y1, Z1) T are defined as: Glass system of coordinates ΩXgYgZg defines the position glass window of the hydrodynamic tunnel. It has the origin in the point Ω, axis Xg and Yg are located in the plane of glass surface, axis Zg is normal to the XgYg plane. Coordinates of origin point Ω and rotation matrix Rg define the position of glass coordinate system in the object coordinate system ( Figure  3 ).
To find image coordinates of point a corresponding to the object point A, it is necessary to determine vectors r 1 , r 2 , r 3 .
Vector r 1 has its origin in the center of projection C. In glass coordinate system vector r 1 is defined by the following equations: The location of the point A 1 in the glass coordinate system is determined using the condition of ZgA 1 = 0: The direction of vector r 2 is determined using Snell law: where n1 is the coefficient of refraction at the air-glass interface, φ1 is angle of inclination of ray r 1 , φ2 is the angle of refraction.
The origin of vector r 3 is point A2, that can be found as: And the direction of vector r 3 is determined in similar way: where n2 is the coefficient of refraction at the glass-liquid interface, φ2 is angle of inclination of ray r 2 , φ3 is the angle of refraction.
The transformation from glass coordinate system to the object coordinate system is given by: For multi media imaging case, equations (1) -(9) serve as analogue of the co-linearity equations for single optical media image acquisition. They allow to establish relation between object point A and its image a taking into account the position of glass system of coordinate XΩg and refraction indexes of the glass and liquid nag, ngw: These relations are used for estimation the unknown parameters of the imaging model using image coordinate of reference points as observations.

Calibration technique
Imaging model (Eq. 10) includes several parameters that are to be determined during the calibration procedure or to be measured by some independent technique. These parameters are: position of glass coordinate system, glass thickness, refraction indices of optical medias. For calibration procedure allowing to acquire a set of test field images in target working space (liquid) (Knyaz et al., 2021a), position of glass coordinate system was included in the vector of unknown parameters and was determined during the developed calibration procedure. The refraction indices were known and glass thickness was measured by precise mechanical tool.
For the case of inaccessible working space (as for hydrodynamic tunnel HDT-400) there is no possibility to estimate glass coordinate system during calibration procedure. Also it is impossible to measure the thickness of the glass wall. To define these parameters, the following technique is developed.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France At first, calibration of the photogrammetric system is performed in single optical media (air) environment (Figure 4), resulting in a set of interior orientation parameters v sm io = (mx, my, bx, by, a0, a1, a2, a3, a4, a5) T . Then calibration procedure is performed in working space of the hydrodynamic tunnel in "dry" mode (working space is not filled with water). During the calibration the glass thickness dg is estimated.
The rest of the unknown parameters -position of glass coordinate system -is determined just before 3D measurements in working mode of the hydrodynamic tunnel. For this purpose a set of coded targets (Kniaz et al., 2021) are placed on the surface of the glass wall, and parameters of exterior orientation v sm eo = (XΩ, YΩ, ZΩ, αΩ, ωΩ, κΩ) T are found using interior and exterior orientation parameters of the photogrammetric system determined at the first stage.

RESULTS AND DISCUSSION
Experimental evaluation of the developed algorithms for system calibration for inaccessible working space in multi-media optical environment was performed using original photogrammetric system and the laboratory setup described in Section 3.1.
The developed calibration technique was evaluated using the following procedure. At the first stage calibration parameters were estimated in single optical media (air).
Then photogrammetric system was installed in the working position required for 3D measurement in laboratory conditions and calibration was carried out in "dry" mode, test field being located in working space of hydrodynamic tunnel. As a result of this phase the glass thickness dg was estimated. Also the position of the glass coordinate system was determined for "dry" mode using a set of coded targets placed on the glass surface.
At the third phase of the evaluation the water tank was filled with water, and calibration procedure was performed for "wet" mode (Knyaz et al., 2021b). At this phase the position of the glass coordinate system was estimated during the calibration procedure, along with the standard set of calidration parameter v sm io = (mx, my, bx, by, a0, a1, a2, a3, a4, a5) T (called as "wet" set in following discussion). The refractive indices for glass and liquid were taken as known for the second the third phases of evaluation.
The results of calibration are presented in Table 1.
To evaluate the quality of the developed calibration technique for inaccessible working space, 3D scanning of reference 3D model (stereolithography model of a wing) was performed in working ("wet") mode. Two sets of calibration parameters were used during 3D scanning: "dry" parameters and "wet" parameters.
"Point cloud"-to-"point cloud" distance (Knyaz and Zheltov, 2017) for the scanned surfaces of the reference object, obtained using "dry" and "wet" sets of calibration parameters, was used as the metric of calibration accuracy.   Figure 6 shows, that surface 3D reconstruction using developed calibration technique for inaccessible working space corresponds to the surface obtained with "wet" calibration parameters set with very high degree.
The evaluation of the developed technique for calibration of the photogrammetric 3D measurement system showed that it provides the accuracy of the calibration at the same level as calibration for accessible working space.