A SIMPLE DEVICE FOR MONITORING CRACKS FROM PHOTOGRAPHS

The article presents a simple technological solution PMC (PhotoMicrometer Contrast) for monitoring cracks and deformation joints in buildings and structures. The solution is based on the methods of image processing and photogrammetry. The monitoring equipment includes a special marker, a digital camera and specialized processing software. The marker consists of two thin plastic plates where Aruco marks and special measuring areas are printed. There are two types of markers – for monitoring in one or two directions of crack development. While monitoring is performed, two plates are fixed on both sides of the crack or deformation joint. Then the plates are photographed with a digital camera as often as the observation cycle requires. Any conventional modern digital camera is suitable for photographing after successfully passing the distortion effect test described in the article. Image processing in the software includes automatic identification of the marker in the image, binarization of the image, determination of its scale and dimensions of measuring areas in the current observation cycle. For processing images made at large angles of inclination and rotation, a preliminary solution of the resection in space and the images rectification to the found angles is provided. The dynamics of crack development is determined by the results of several cycles of observations collected over period. Our technology ensures the accuracy of the determination of coordinates and deformations at the level of 0.05 mm and higher for photographing distances from 0.3 to 50 m.


INTRODUCTION
One of the important tasks of engineering inspection of buildings and building constructions is to monitor the dynamics of cracks development. In addition to other systems, devices and technologies based on the use of photographs and photogrammetry methods have been significantly developed in recent years. Many researchers pay attention to solving the problems of detecting and measuring cracks (Hampel, Maas, 2009;Rau et al, 2017;Sarker et al, 2017;Valença et al, 2013;Zheng, 2014). Studies on the use of photogrammetry methods for crack monitoring are also being actively carried out (Barazzetti, Scaioni, 2009;Nishiyama et al., 2015;Bal et al., 2021). In our opinion, the most effective of them are based upon the use of deformation markers with marks automatically recognized using specialized photogrammetric software (Germanese et al., 2018;Wojnarowski et al., 2019). This approach makes it possible to completely eliminate the "human factor" from the measurement processes and to develop universal crack monitoring technologies, including fully automated ones. In this article, we present probably the simplest device and the technology for monitoring the crack opening value using photographs.

METHOD
The monitoring technology assumes using marker, digital camera and PhotoMicrometer C processing software. The marker can be of two types: for one-dimensional ( Fig.1) and for twodimensional ( Fig.2) determinations.
We are presenting the device and the principle of operation of the marker on the example of a simpler model. The PMC marker (PhotoMicrometer Contrast) consists of two plates made of white plastic with a 0.5-0.8 mm thickness. On both plates, Aruco marks and measuring areas are printed in black. The measuring areas are designated as C1, C2, C3 and C4 in Fig.3.

Figure 3.
Wide and narrow marker plates with the designation of measuring areas.
All areas have the same height. С1 and С2 have the same width also. In this study, the marker with the size of a wide plate 50x90 mm was tested, and the size of areas C1 and C2 was 15 mm. To monitor the crack, the marker is installed as shown in Fig.4. As can be seen in the picture, the wide plate has slots where the narrow plate is threaded in such a way that parts of the measuring areas C3 and C4 get hidden under the wide plate. Wide and narrow plates are fixed (glued) on different sides of the crack under study.
The principle of operation of the marker is that when the crack width increases (decreases), the width of the visible part of the areas C3 and C4 will change. At the same time, the width of areas C1 and C2 does not change and their dimensions are used as reference. The current position and the size of the measuring areas can be fixed by photographing the marker. Using the known initial sizes of the C1-C4 areas and the dimensions of their images measured in the image, it is possible to determine accurately the current width of the C3 and C4 areas in millimeters, even without resorting to complex photogrammetric calculations.
The vernier scale applied to the PMC-1D marker is additional and expanding the functionality of this device.

Digital camera
Conventional modern digital cameras can be used for monitoring with a PMC marker. When choosing a camera for monitoring, an important requirement is the ability to take detailed photos from different distances, so it is convenient to use telephoto lenses with variable focal length and telecompact cameras. The dependence of the photographing distance and the equivalent (in terms of the 36x24 mm frame format) focal length of the camera is shown in This table is calculated for the marker of the above dimensions and for a camera with a resolution of 15 megapixels. For cameras with a different resolution, the focal length values shown in the table will differ. For example, for a camera with a resolution of 50 megapixels, the corresponding focal length values will be approximately 2 times less. There are also some other criteria that it is desirable to observe when choosing a camera for monitoring (PHOTOGRAMMETRIA, 2022).
In any case, before photographing, the camera must be tested for the effect of distortion as shown in Section 3.1.

The PhotoMicrometer C software
The PhotoMicrometer C software was specially developed to operate with the markers of this design and allows to perform fully automated processing of images from mark measurements up to the creation of monitoring reports. The main functions of the program are as follows: -uploading a photo(s) taken during the current monitoring cycle and automatically finding the marker image in the photo(s) ( -calculation of the X variable for the monitoring cycle and an accuracy assessment: Where Si = the sum of the black pixels of the i-th measuring area Ci (calculated automatically after marker image binarization); L = the real width of the С1 and С2 areas in millimeters. If the observations in the cycle consist of several (n) photos, then the X count and the RMS error mx are calculated as follows: If there are results of several cycles of marker observations, the software calculates ∆ differences between the X variables of each cycle and the initial one. The differences reflect the dynamic of the crack at the marker installation site. Monitoring results can be displayed in the form of tables and graphs.

TESTS, RESULTS AND DISCUSSIONS
Following criteria were used to test the system functioning: -the distortion impact onto the measurement results; -the photographing distance impact onto the measurement accuracy; -the photographing angle impact onto the measurement results.

Study of the distortion impact
Unlike the Photomicrometer system developed for 3D monitoring (Wojnarowski A.E. et al., 2019), the presented PMC system does not have a built-in function for detecting and accounting for camera distortion. However, the effect of distortion on measurements and calculations performed according to the algorithm described above and related to the definition and accounting of the image scale turns out to be significantly less than it is in systems where photogrammetric coordinates are directly measured and used. Images of PMC markers taken by cameras with medium-focal-length and longfocal-length lenses, as a rule, can be processed in the PhotoMicrometer C software and give high-quality results without entering corrections for distortion.
On the other hand, distortions of the optical system can be easily eliminatedmany modern cameras have either a built-in function to eliminate lens distortion or this function has a standard software supplied by the manufacturer with the camera. Nevertheless, when choosing a camera for monitoring, it is important to be able to check the effect of distortion on the results of measurements performed using the PMC system. To solve the task, we offer the following test, which we will demonstrate using the example of a Canon EOS 70D camera with a lens supplied in the kit: Kit EF-S 18-55. To perform the test, it is necessary to take 5 photos of the PMC marker in different parts of the frame (center, left, right, top, bottom) as shown in Fig. 7. For the experiment, the photographing was performed twice with the extreme values of the focal length (f = 18 mm and f = 55 mm) at a distance of 0.3 m and 1 m respectively. This camera does not have a built-in distortion elimination function, so the photos were simultaneously saved in two RAW and JPG formats (RAW+JPG menu settings). The photos saved in RAW format were corrected for distortion at a result of post-processing in the standard Canon Digital Photo Professional (DPP) program. And the photos originally saved in JPG format were not corrected for distortion. In order to understand what distortion values we are dealing with, the camera was previously calibrated in a third-party photogrammetric program with the preservation of the camera settings described above. The maximum distortion values obtained from calibration are shown in The maximum values of radial distortion in the images before and after its correction in the Canon DPP program.
As you can see in Table 2, when a focal length is f = 18 mm radial distortion at the edge of the frame achieves more than 162 pixels and the standard program does not completely eliminate it; which is understandableeach lens is unique, and the program supports averaged distortion profiles for a lens type. The Table also shows that when a focal length is f = 55 mm the distortion is insignificant.
Thus, as a result of photographing the PMC-1D marker according to the scheme shown in Fig.7 and eliminating distortion in the standard Canon DPP program, we have 4 types of images: 1. From a distance of 0.3 m and f = 18 mm without the distortion correction. 2. From a distance of 0.3 m and f = 18 mm after the distortion is corrected. 3. From a distance of 1 m and f = 55 mm without the distortion correction. 4. From a distance of 1m and f = 55 mm after the distortion is corrected.
Each of these variants contains 5 photos taken in accordance with Fig. 7. The results of these photos processing in the Photomicrometer C software are shown in Fig.8. -Fig.11. in tabular and graphical form. As can be seen in Fig. 8 a distortion of 160 pixels and more leads to a maximum spread of X-values equal to 0.259 mm between the images taken on the left and right. At the same time, the average value of X counts is -4.337 mm, and the RMS deviation of the counts is 0.083 mm. Figure 9. The results of processing photos taken from a distance of 0.3 m and f = 18 mm after the distortion is corrected.
As can be seen in Fig. 9 after the distortion is corrected, the residual limit distortion of 17.8 pixels leads to a maximum spread of X values equal to 0.048 mm. The average value of X counts is -4.298 mm, and the RMS deviation is 0.016 mm. The results of photos processing shown in Fig.10 have an amplitude of X values equal to 0.040 mm. The average value of X counts is -4.290 mm and the RMS deviation is 0.013 mm. Figure 11. The results of processing photos taken from a distance of 1m and f = 55 mm after the distortion is corrected.
The amplitude of the X values shown in Fig.11. is 0.038 mm. The average value of X counts is -4.304 mm and the RMS deviation is 0.014 мм.
As can be seen from the results obtained, only in the first case, when photographing at a wide angle, the effect of distortion is significant. But the use of a standard distortion correction program solves this problem. Photographing at a focal length f = 55 mm gives good results both with and without correction of distortion. It should also be noted that the average values of the X counts differ insignificantly in all cases.
Thus, despite the long description, the algorithm for controlling the effect of distortion is quite simple: you need to take a picture of the PMC marker, as shown in Fig.7, process the photos in the Photomicrometer C software and calculate the amplitude of the counts. For good lenses this value as a rule does not exceed 0.05 mm.
Usually problems with distortion can occur when photographing with wide-angle lens. Photos taken with medium-focus or longfocus lenses, as a rule, have a slight distortion and can be processed without correcting it. All further experiments, the results of which are given in this article, were performed without correcting of distortion.

Study of the photographing distance impact
For this study, a marker was fixed on a mechanical micrometer with a 0.01 mm scale division (Fig.12). The marker was photographed with a Nikon Coolpix P900 camera sequentially from distances of 1, 10, 20, 30, 40 and 50 meters with equivalent focal lengths from 100 mm to 2000 mm. Then an imitation of the crack opening by 0.5 mm was performed on the micrometer scale, and the photographing was repeated in reverse order. As result, 5 images were taken at each position. The processing results are shown in the Fig.13.   Figure 13. A graph of the mutual position of the two plates relative to the initial one based on the results of processing images taken from different distances.
As it is shown in the figure, a shift of 0.5 mm is clearly legible, and the maximum spread of values is observed at a maximum photographing distance of 50 m (the amplitude is 0.13 mm, the RMS deviation is 0.048 mm). At other distances, these values are significantly smaller.

Study of the photographing angle impact
For this study the marker with a constant position of the measuring areas was photographed at different angles of inclination and rotation (Fig. 14). As can be seen in Fig.14, totally 19 photographs were taken in this study. The top row shows one photo made perpendicular to the plane of the marker. The second row shows photos taken at different values of the horizontal angle in increments of 10 degrees (from 10 to 60 degrees inclusively).The third one presents photos with different values of the angle of inclination, also in increments of 10 degrees (from 10 to 60 degrees). And in the fourth row, photos with different angles of rotation in the plane of the wall are presented in a similar way. The photographing was performed from a distance of 2 m with Canon EOS 70D camera. An EF 70-200 lens and a focal length f = 135 mm were used.
The software Photomicrometer C allows you to perform calculations in two ways: -Binarization and counting of black pixels are performed on the original image, then calculations using formulas (1) and (2) are performed; -The resection in space is solved beforehand and the image is rectified to the found Eulerian angles. In this case, the rectified image is processed according to the algorithm described above.
The interfaces of these solutions for the image with an angle of inclination 50 degrees are shown in Fig.15, on the left -the solution according to the original image, on the right -after prior image rectification: The results of processing all the images without prior rectification are shown graphically in Fig.16. The sequence of values in the graph shown in Fig.16 corresponds to the sequence of images in Fig.14. The values corresponding to horizontal angles (from 10 to 60 degrees) are highlighted in red, the values corresponding to the angles of inclination are marked in green and the values corresponding to the angles of rotation in the plane of the wall are blue. As you can see on the graph (Fig.15), there is a little systematics in the results corresponding to horizontal anglesthe counts increase with increasing rotation angle. Other angles do not have a noticeable systematic effect on the results. The amplitude for all values was 0.096 mm, and the RMS deviation was 0.027 mm.
The following graph (Fig.17) shows the results of processing the same images after performing their prior rectification. Figure 17. Graph of the results of images processing after prior rectification.
As it is shown in the figure, the use of image rectification to angles obtained as a result of solving the resection in space, leads to a decrease in the systematic influence of horizontal angles and generally improves the results (the amplitude for all values was 0.060 mm, the RMS deviation was 0.012 mm).
The systematic errors shown arise due to differences in the image scales at the region of reference areas C1 and C2 (Fig.3.) caused by the horizontal angle of the camera rotation. The magnitude of errors caused by the different scales of the image also depends on the photographing distance: the greater the distance, the smaller the differences in scales and the smaller the error data. As our studies have shown, at photographing distances of more than 5 meters, errors caused by the different scale of the image are practically not noticeable. There are a couple more important points related to the quality of manufacture and installation of the marker, which affect the ability to correctly process images taken at large angles of inclination and rotation. Firstly, the slots on the outer plate of the PMC-1D marker should be made with a bevel so that the outer plate does not cover the inner (areas C3 and C4) at large horizontal photographing angles. Secondly, the marker must be manufactured and installed so that all measuring areas C1 -C4 geometrically represent parallel regular planes, as far as possible.