ORTHOPHOTO SHADOW DETECTION METHOD UNDER ARTIFICIAL SHADOW

: Shadows are ubiquitous in high-resolution images, especially in urban regions where there are more serious shadow occlusions. In order to improve the detection effect of shadows, this paper analyzes the characteristics and properties of shadows in orthophotos, and proposes an orthophoto shadow detection method under artificial shadow. Firstly, the shadow modeling tool is used to calculate the shadow regions (i.e. artificial shadow) caused by the building obstructing the sun's rays. Secondly, the relaxation matching algorithm is extended by the position and the shape of the shadow polygon as characteristic constraints. The relaxation matching algorithm is extended by the position and shape as shadow polygon’s characteristic constraints. Thirdly, the feature constraint value is calculated which between the shadow polygons of the two images. The correlation coefficient is used to obtain the initial probability value of each shadow polygon in the orthophoto. Finally, the optimal solution is obtained by continuous correction and iteration of the initial probability value. The method performs an overall matching of the two images and obtains the position of the shadow regions of the buildings in the orthophoto image. Experiment shows that this method reduces the mismatch rate and improves the matching accuracy. This method can detect shadow regions of buildings in orthophoto quickly and efficiently.


1． INTRODUCTION
In the process of aerial image, objects such as tall buildings and trees have a serious impact on the image quality. Therefore, there are always shadow regions formed by tall buildings, trees, etc., which are blocked by light source. The presence of shadows restricts the effective use of aerial images. Shadow is a ubiquitous image feature in remote sensing images. Shadow is a common image feature in remote sensing images (Zhou, 2017a). The existence of shadow brings great difficulties to the subsequent processing of images such as image recognition and classification. Therefore, it is an urgent problem how to accurately detect the shadow regions and eliminate the shadow in the image. And it is also a difficult problem in the remote sensing image preprocessing work.
Until recently, many scholars have analyzed the characteristics of shadows to propose a number of related algorithms for detecting shadow regions. Xie and Zhou (2008) proposed to use the geometric model of buildings to calculate the shadow regions of buildings in urban areas. It is proposed that shadow detection method based on digital building model (DBM) (Xie, 2010). The proposed black-body radiation model is used to detect shadows. This method is completely driven by the physical process of shadow formation. All parameters are directly calculated (Makarau, 2011). Liu et al. (2012) adopt the object-oriented classification method formed by the relationship between shadows and the non-shadows of adjacent regions. This method can effectively detect the shadow region in the image. Zhao et al. (2014) proposed a new color model based remote sensing image shadow detection method to improve shadow detection accuracy. An edge-based soft boundary model that uses image brightness to detect boundary shadows (Dong, 2014). The shadow regions are detected by hyperspectral image segmentation and mathematical statistical correlation method (Gomes, 2016). Bargoti and Underwood (2017) proposed a pixel-wise approach for image segmentation using convolutional neural networks with multi-scale and multi-layered perceptron to respond to the change in illumination conditions. Suh, Hofstee and van Henten (2018) developed an algorithm based on a multi-level threshold and color space conversion for detecting and removing vegetation shadows on the ground. By analyzing the above methods, there are many the following shortcomings in the existing shadow detection algorithm: 1) It always needs to rely on prior knowledge for model-based shadow detection algorithm.
2) The selection of threshold requires a strong prior knowledge. It can be adjusted continuously to achieve satisfactory results and reduce the applicability of the detection method according by human experience.
3) There is no balance between the practicability, accuracy and complexity of some algorithms. For example, some algorithms are very practical, but they are too complex to be applied to practical projects.
According to the shortcomings of the above methods, this paper uses the accurate building information provided by the image DBM to generate the building reference shadow image and the overall matching of the shadow regions of the buildings in the two images. In order to reduce mismatch and improve matching accuracy, this paper proposes an orthophoto shadow detection method under artificial shadow.

2． ORTHOPHOTO SHADOW DETECTION METHOD
This paper proposes an orthophoto shadow detection method under the artificial shadow channel by taking the shadow elements of buildings as the experimental objects. Firstly, the method uses the known elevation angle and azimuth of the sun to calculate the geometric range of the shadow regions caused by the building's obstruction of the sun's rays on the ground plane (i.e., artificial shadow) by the shadow modeling tool. Then, the edge detection algorithm is used to extract the contours of the image. The extracted data is segmented by combining the DBM data to obtain the edge data of the architectural shadow. Finally, the shadow polygons in the two images are matched according to the overall slack matching method of the polygon features. The final shadow detection result is obtained.

Matching
Assume that the two polygons of the planar building are to be matched with the data Where and are the building shadow polygons to be matched in the data 1 , 2 , respectively; m and n are the number of polygons corresponding to each data set, respectively. The general matching method is to use each matching indicator separately, and calculate the similarity of each index in order. This matching strategy determines the threshold for each indicator. And the order in which the indicators are used is also important. The matching results obtained by different index sequences may be different. Therefore, these indicators should be used as a whole to calculate a total similarity value to determine the matching result. The difference index describes the difference between two candidate polygons. The evaluation target difference index includes polygon position, shape, size and so on (Zhou, 2017b). In this paper, the positions and shapes of the matching pairs are used to measure the feature difference between the candidate polygons. Then, the difference index between the polygons is calculated. The initial probability matrix is synthesized.
According to the assumption that the data to be matched represents the set of coordinates of the center of gravity of the polygon, and C represents the set of the perimeter of the minimum outer rectangle of the polygon. In this paper, the geometric similarity between and corresponding polygons is calculated respectively, and the calculation formula is as follows: Where and represent arbitrary polygons in the two sets of data, respectively, and are the position, area and shape similarity of the candidate matching pairs are the barycentric coordinates of the polygons and . Combine these two features with a polygon as a matching unit. Correlation coefficients were calculated by calculating each polygon in the data 1 and all candidate matching polygons in the data 2 . The calculation formula is as follows: Where 1 , 2 are the weights of position and shape similarity, respectively. By combining multiple differences indicators, the initial probability of the composite is obtained by calculating the matching probability of the two characteristic indicators and weighted average. The weights of 1 ,and 2 are both 1/2. There are M building shadow polygons in the data 1 , and the corresponding polygons are found from the data 2 (including N building shadow polygons). Based on the eigenvalues of the polygons, a similarity matrix R for each polygon of the two sets of data is calculated.
Firstly, the initial candidate matching pair is obtained by setting the corresponding threshold of position and shape similarity. The candidate matching pairs with obvious errors are eliminated, the similarity less than the given threshold is deleted and the value is assigned to 0. The final similarity matrix R′ × is obtained. Then, in order to conduct relaxation optimization calculation, the initial probability of matching m polygons in the data 1 is estimated. That is, the feature correlation coefficients of each candidate surface are added, and the correlation coefficient sum of all candidate faces is divided as the initial probability. The calculation formula is as follows: Finally, the initial matching probability of each element in the matrix R′ × is calculated according to formula (4), and the initial matching probability matrix is calculated. Where, each row represents the initial probability of polygon in data 1 and n polygons in data 2 .

Pairs
The initial matching probability only reflects the local consistency between the candidate matching pairs, and does not consider the neighborhood context structure features. The matching probability of spatial targets is often affected by the matching degree of other surrounding entities and the characteristics of neighborhood structure (Zhang, 2014). Therefore, it is necessary to consider the relative geometric relationship and matching degree of adjacent candidate matching pairs, and heuristically correct the initial matching probability based on local similarity. There is no explicit neighborhood relationship between polygon targets. The number of target face neighborhoods of the building shadow polygon data S and the data T is determined by the breadth-first search strategy with the center of gravity of the shadow region as the search target.
After the neighborhood relation is determined, the characteristic difference value and support degree between the neighboring candidate matching pairs are calculated, and the initial matching matrix based on local geometry is updated heuristically. Let ℎ ∈ ℵ( ) , ∈ ℵ( ) , then ℎ and represent the polygon targets in the neighborhood of and , respectively, and ( ℎ , ) are the adjacent candidate matching pairs of � , �. Since the neighborhood elements may contain a single target and a neighborhood pattern composed of multiple targets, there are repeated calculations when calculating the support coefficients of each neighborhood element. Therefore, it needs to be combined. The relative position and relative shape relationship between � , � and ( ℎ , ) are calculated separately. The calculation formula is as follows: Where, 1 � , ; ℎ , � , 2 � , ; ℎ , � represent the relative position and relative character relationship between � ℎ , � and � , �, respectively; ( , b) represents the distance between the centers of gravity of the two polygons a and b; ′ ( , b) represents the difference between the circumference of the minimum outer rectangle of two polygons a and b. Combining the above two relative geometric relations, the compatibility coefficient ( , ; ℎ , ) between � ℎ , � and � , � is obtained as: Where 1 , 2 , and 3 are the weights of position, shape similarity, and correlation coefficient, respectively. Their values are all 1/3.
Calculate the support coefficients of each neighbourhood shadow polygon ℎ ∈ ℵ( ) of according to equation (8), and divide the support coefficients of each neighborhood element into two sets After adding three feature constraints, the iterative update formula of the matching probability can be expressed as: The iteration stops when the change probability of the matching probability of each candidate matching pair is less than a given threshold during the two iterations. According to the convergence probability matrix, the final matching result is selected, and the candidate matching pairs with the highest matching probability in each row and column of the probability matrix are selected. When there is a common polygon target with other matching pairs, the geometric similarity between the two sets of matches before and after merging is calculated. If the similarity after the merger is greater than that before the merger, the two are merged to match many-to-many. Otherwise, the matching pair with higher probability is retained, and the lower probability is the matching target or the adjacent mode with the second probability.

3． EXPERIMENTAL RESULTS AND ANALYSIS
In order to verify the reliability of the method, this paper selects the high-resolution aerial image of the city of Denver in the United States as experimental data. The buildings in the image are complex and diverse. Since the shooting height is relatively low, the position far from the centre of the photograph has a large degree of tilt, resulting in a wide range of occlusion.
1) Image data: The aerial image covers a city area. The average altitude of the aircraft is 1650m, of which the heading overlap rate is 65% and the side overlap rate is 30%. The image obtained by orthorectification of the original image is shown in Figure 1.
2) DBM data: Digital building model (DBM) is a corresponding relationship with aerial imagery. It is a three-dimensional information composed of the two-dimensional coordinates of the building and the height of the building (Figure 2).  Because the building information of the DBM data in the experiment is not complete enough, some regions in the generated artificial shadow image are blank. Therefore, this article only performs shadow detection on the area where the shadow of the building is generated. By superimposing the shadows of the buildings in the two images, it is found that the artificially generated shadows overlap with the shadows in the orthophotos. However, there is a certain offset in its position.
According to the above problems, this paper uses the global relaxation matching method based on polygon features to match the building shadows in two images. Through this method, more accurate shadow regions of buildings can be obtained. In this paper, the centre of mass of the shadow polygon is expressed as the polygon, and the boundary contour of the building shadow is extracted by the edge extraction algorithm. According to the building contour obtained from DBM data, the shadow of the building is further segmented. Analysing the similarity value of shadows of each building and determining two indices of set similarity evaluation: position and shape similarity. Using these two indicators to determine the matching relationship between building shadows. Then, the eigenvalues of each of the shadow polygons are calculated to determine the initial matching matrix. Finally, the initial matching probability matrix is iteratively updated until the matrix converges by combining the relative geometric relationship and matching degree of the adjacent candidate matching pairs. The matching pairs are determined by this method. The image local matching results are shown in Figure  4.
(a) (b) Figure 4. Local shadow matching image After many experiments, the similarity evaluation threshold is set to 0.7, and the desired matching pairs can be selected. Table  1 gives the matching relationship and similar values of the shaded polygons in Figure 4. The shadow polygons of the seven buildings in Figure 4a determine the matching pairs, indicating that the proposed method can accurately identify the shadow matching relationship between the two images. Since the image has a shadow occlusion problem during shooting, the position and shape of a4 and b4 are largely different. Since there is occlusion of the building shadow in Figure 4a, no matching pairs with b9 and b10 in Figure 4b are found in Figure 4a. The experimental results show that the matching success rate is higher in the case where the matching data has a positional offset but the phase accuracy is high. Therefore, the feasibility of the method is also verified. The resulting experimental result is shown.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3/W10, 2020 International Conference on Geomatics in the Big Data Era (ICGBD), 15-17 November 2019, Guilin, Guangxi, China Figure 5. Shadow detection result

4． CONCLUNSIONS
This paper proposes an orthophoto shadow detection method based on artificial shadow channel. The method mainly obtains the exact position of the building shadow in the orthophoto by loosely matching the artificially generated building shadow with the shadow in the orthophoto. This method avoids the loss of shape information according to the shape description function and is invariant to rotation, translation and expansion.
The experimental results show that the method can more directly determine the position of the architectural shadow and more accurately capture the area of the building shadow. Due to the occlusion of some building shadows during shooting, there is a large difference between the shadows in the orthophotos and the artificially generated shadows. In the next work, it is still necessary to further refine the artificially generated shadows.