3D RECONSTRUCTION OF ICE SHAPE USING VISIBLE AND THERMAL RANGE IMAGING FOR AIRCRAFT ICING STUDY

Aircraft icing is one of the main factors decreasing the flight safety. Qualitative and quantitative understanding of the icing process is crucially needed for developing anti-icing measures and safety recommendations. Changes in aerodynamic characteristics of aircraft caused by changes in shape of aircraft surfaces due to the ice accretion can lead to significant aerodynamic performance degradation. So the reliable and accurate information of how the shape of the ice accretion influences on aerodynamic characteristics is a key point for predicting the changes in aerodynamic performance. The study addresses to a problem of accurate shape measuring of ice accretion for further experimental study of iced-aircraft aerodynamic in a hydrodynamic tunnel. For this purpose the evaluation of various techniques of ice 3D measurements is performed that include as visible so thermal imaging of ice accretion. The results of evaluation serves for the decision of preferable technique to be used in experimental study. Also the framework is developed for creating physical models of iced aircraft based on result of real ice accretion shape measurement. It allows to produce stereolithography (SLA) copies of of an aircraft under icing condition for different levels of icing. These SLA-models of an aircraft under icing condition are then used for flow behaviour study in order to identify critical flying condition.


INTRODUCTION
The study of aircraft icing process is an essential part in providing the safety of flights. Such study includes two parts: (1) the study of aircraft icing process that gives information of how icing is developing in time under the given conditions and (2) the study of how icing of various shape and degree influence on aerodynamic characteristics of a plane.
Due to extreme difficulty of studying icing process in real aircraft flight, most of the experiments is carried out in aerodynamic or hydrodynamic tunnels. They allow to model given flight conditions and to monitor and control the flow process. The adequacy of the results obtained is based on a similitude concept saying that a model correctly reflects real application if geometric, kinematic and dynamic similarity is observed.
The first part of aircraft icing study is usually performed in icing wind tunnels (Lee et al., 2014), that provide controllable conditions to study ice accretion developing. At this stage ice shape and environment parameters are registered.
At the second stage the registered ice accretion shapes are used for numerical simulation of flight or for physical experiments in aerodynamic or hydrodynamic tunnels. These experiments allow to identify critical conditions for flight safety causing by icing.
This research is a part of a long-term research project aimed at exploring an influence of aircraft icing on flying characteristics * Corresponding author of aircraft by flow modeling in low-speed hydrodynamic tunnel. At this stage of the project different techniques for ice shape 3D reconstruction are evaluated to chose the most appropriate one for the considered task.
The paper presents the results of the experimental study of different technique for ice shape 3D reconstruction and the developed framework for registering icing process and re-producing it in stereolithography models for studying changes in aircraft aerodynamics by experiments in hydrodynamic tunnel.
The main contributions of this paper are (1) experimental evaluation of applicability thermal imaging for measuring 3D shape of ice; (2) comparative study of accuracy of ice shape reconstruction for different types of illumination condition; (3) a pipeline for iced aircraft model producing for aerodynamic study in hydrodynamic tunnel.

RELATED WORK
The problems of aircraft icing attract a lot of scholar attention due to the importance of timely identification of the icing process and understanding of its influence on aerodynamic performance. The area of the study includes the analysis of the state of the air that causes icing (Cao et al., 2018), in-flight icing detection (Ikiades et al., 2007), measuring the shape of ice accretion and its impact on aerodynamic characteristics.
The review (Cao et al., 2018) provides the systematic analysis of the recent researches on aircraft icing. The study considers the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) causes of icing and their relation with environmental and aircraftrelated parameters, types and severity of aircraft icing, and icing impact on aircraft performance. The analysis and summary of influence of different effects of ice accretion on aircraft flying characteristics are given, along with the survey of modern techniques for ice accretion measuring and predicting. Also, the review of aircraft flight accidents provoked by icing is performed.
In-time ice accretion detection on the surfaces of aircraft is crucially important for flight safety. Traditional techniques for ice detection measure the temperature of ambient air and the quantity of liquid water in the air to make decision of icing condition. The approach (Ikiades et al., 2007) proposes to analyze an optical diffusion of light in the ice volume for detecting ice and determine its type. The performed experiments in measuring light diffusion have shown that the type of the ice can be recognized as a result of analysis of spatial distribution of the specular and scattered components of light. Also the proposed technique can be exploited for estimating the thickness of the ice accretion.
But not only the thickness, but also mainly the shape of the ice accretion significantly affects the aerodynamic characteristics of the aircraft. Ice shape measuring is necessary for aircraft icing studying as by experiments in aerodynamic tunnels so by numerical simulation. Various techniques are used for ice 3D shape reconstruction. They can be divided into two groups: contact measurements and non-contact measurements.
For ice shape 3D reconstruction as contact so non-contact techniques are applied. Main disadvantage of contact methods (such as e.g., mold and casting) in measuring ice shape is interrupting the icing process, thus preventing its registration in time. Among the non-contact measurement techniques such as optical, ultrasonic (Hansman et al., 1988) and microwave, the former have a significant advantage in density and accuracy of measurements. The main problem for optical techniques in measuring ice accretion is reflective properties of ice surface.
A set of researches has been aimed to 3D measuring of ice shape for further experimental study of aerodynamics change in aerodynamic or hydrodynamic tunnels (Broeren et al., 2010), (Pouryoussefi et al., 2016), (Lee et al., 2019).
Ice-accretion models (casts) were used for evaluating the aerodynamic accuracy of ice accretion simulation (Lee et al., 2019). These model were produced from 3D surface models obtained by laser scanning technique. Experiments were performed in the NASA Icing Research Tunnel using an 18-inch chord, 2-D straight wing with NACA 23012 airfoil section. Ice accretion shape was acquired for six ice-accretion cases, which then have been used for creating hard copies of the accretion. The experiments were carried out with applying of special opaque spray to provide Lambertian reflectivity of ice surface.

PROJECT OUTLINE
The current work is a part of the research project (Knyaz et al., 2020) aimed at experimental studying of aircraft icing. The main problems to be investigated are (1) accurate 3D measuring of flow in a hydrodynamic tunnel (HDT) for the impact of aircraft icing study; (2) aircraft icing process modelling for experiments in a low-speed hydrodynamic tunnel; (3) flow registration for iced surfaces condition and its accurate 3D reconstruction.
The project framework is shown in Figure 1.  3D model of the part with ice accretion (and its hard SLA model), that allows to model flow behaviour on iced surface. The current work is addressed to developing optical 3D measurement techniques for accurate 3D reconstruction of the ice accretion shape.
Finally, experiments are performed with the SLA model of an iced surface in the low-speed hydrodynamic tunnel, that serve for registering and 3D measuring of the flow disturbed by ice accretion.

EXPERIMENTAL SETUP
To provide comparable results of various 3D scanning techniques for all imaging/lighting configurations laser triangulation technique was chosen as 3D reconstruction method. The working space of 3D measurements was 300 × 300 × 300 mm.
For ice accretion shape 3D reconstruction several configurations of photogrammetric 3D scanning system were used, based as on visible range camera so on thermal range camera.

Thermal range configuration
In thermal range configuration MH SM567 thermal camera serves for image acquisition and Iradion® Infinity CO2 laser was used for object lighting. Main specifications of the thermal camera and long-wave infra red laser are presented in  Table 2, and specifications of laser diode modules are given in Table 3. 3 Beam divergence, mrad < 2 < 2 < 0.6 Rise Time, µsec < 5 < 15 < 5 Power stability from cold, % ±5% ±5% ±5% Table 3. Main characteristics of laser diode modules

Calibration
The calibration of the 3D measuring system is prerequisite stage providing the required accuracy of the measurements. The basic imaging model for camera calibration is central projection, expressed in form of co-linearity equations describing the relation between the object point S, the corresponding image point s and the center of projection C: Two systems of coordinates are used to define the relation between the object point S and its image xs: (1) object coordinate system OXY Z that is related with the object and (2) image coordinate system Cxyz that is related with image plane. In Eq. 1 the object point S and the projection center C are determined in OXY Z system of coordinates, and the image point xs -in Cxyz system of coordinates.
The transformation of digital (pixel) image coordinates x d s , y d s of the point S to its image coordinates xs = (xs, ys, zs) T is defined by the following Eq. 2 -4: For accurate image-based 3D measurements the basic imaging model (1) should be modified for taking into account distortions introduced by real imaging system. The correction of the distorted image coordinates (x, y) to obtain undistorted coordinates (x,ỹ) can be written as:x = x + ∆x; (5) y = y + ∆y; with (∆x, ∆y) being additional terms describing various distorting factors.
The parameters of the distortion model are determined as a result of calibration procedure, starting with acquiring a set of test field images with known coordinates given reference points. Then least mean square estimation is used for determining unknown parameters of the distortion model, considering image coordinates of test field reference points as the observations.
The vector of unknown parameters of the interior orientation v l e = (xp, yp, mx, my, a0, ..., a5) T consists of coordinates of principal point, image scales and additional parameters correspondingly, spatial coordinates of reference points being known by independent precise measurements.
The same flat test field with 49 reference points was used for both (visible/thermal) cameras calibration. Sample images of the test field in visible and thermal range acquired by evaluated cameras are shown in Figure 5. Series of 15 images of the test field in different positions and orientations relatively to the camera were acquired by both visible and thermal range cameras. Automatically detected reference points in the images were considered as observations for original calibration procedure (Knyaz, 2006, Knyaz et al., 2021 for estimating parameters of the distortion model (Eq. 7 -8).
The results of camera calibration for visible and thermal cameras are shown in Table 4

EVALUATION RESULTS
First experiments were performed with ice structure to obtain qualitative estimate of possibilities to use structured lighting of different kinds. Figure 6 presents sample images of an ice object in visible range with laser module of red light (650 µm) and the same ice object in thermal (long wave infra-red 8 . . . 14 µm) range with structured light of 9.3 µm. From the other hand thermal lighting is suitable for detecting structured light in the thermal image ( Figure 6(b)), and consequently, to 3D reconstruct the shape of the ice structure without any additional measures.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) (a) ice structure vision range image with thermal (9.3 µm) structured light (b) ice structure visible image with red (650 µm) structured light) Figure 6. Ice structure images in visible and thermal range At the second stage of the experiments, an ice accretion was formed on the surface of the wing model (Figure 7). This ice model was also scanned by vision and thermal system configurations. From the other side, thermal system configuration demonstrated worse accuracy and resolution for diffusely reflecting surface, but does show capability 3D reconstruct ice surface with the quality reasonable for studying aircraft aerodynamics degradation due to icing (Figure 8(c)).
Preliminary experimental study has demonstrated that thermal 3D scanning can be applied for ice accretion 3D measurements, providing reasonable quality of the resulting 3D models. But the thermal 3D scanning configuration has notable drawbacks comparing with vision range configuration. These drawbacks are relatively high cost of the equipment and the complexity of maintenance.
Meanwhile the applying vision range configurations along with using opaque spray is more preferable for ice accretion 3D reconstruction at the first stage of the study.

CONCLUSION
Experimental evaluation on test surfaces having diffuse reflectance has shown that optical 3D measurements in long-wave infra-red range have lower accuracy and resolution that visible range 3D scanning.
Meanwhile, thermal 3D scanning allows to reconstruct 3D shape of icy surfaces without applying the opaque spray, thus providing a possibility of 3D registering the process of ice accretion evolution in time on aircraft surfaces. Also thermal 3D scanning provides sufficient accuracy and resolution to be applied in aircraft icing study.