A Novel 3D Building Damage Detection Method Using Multiple Overlapping UAV Images

In this paper, a novel approach is presented that applies multiple overlapping UAV imagesto building damage detection. Traditional building damage detection method focus on 2D changes detection (i.e., those only in image appearance), whereas the 2D information delivered by the images is often not sufficient and accurate when dealing with building damage detection. Therefore the detection of building damage in 3D feature of scenes is desired. The key idea of 3D building damage detection is the 3D Change Detection using 3D point cloud obtained from aerial images through Structure from motion (SFM) techniques. The approach of building damage detection discussed in this paper not only uses the height changes of 3D feature of scene but also utilizes the image's shape and texture feature. Therefore, this method fully combines the 2D and 3D information of the real world to detect the building damage. The results, tested through field study, demonstrate that this method is feasible and effective in building damage detection. It has also shown that the proposed method is easily applicable and suited well for rapid damage assessment after natural disasters.


Introduction
In recent years, the reported occurrences of natural disasters, such as earthquakes, floods, tsunamis, and mud-rock flow, are on the rise, leading to increased public awareness of the impact of catastrophic events, such as the 2008 Wenchuan earthquake, the 2010 Chile earthquake, the 2011 Japan tsunami and the 2013 Ya'an earthquake.After natural disasters, rapid damage assessment is crucial for initiating effective emergency response actions.Building as a shelter is an essential place for human, estimating building damage due to natural disasters in real-time and scenario mode is becoming more necessary as emergency response and reconstruction after the disasters.Space borne and airborne remote sensing has been an important tool used for recognizing, rescuing, recovery and managing tasks in the event of a disaster.Therefore, it is important to detect the building damage from the remote sensing image for estimating building damage.
Nowadays, researches have proposed many building damage detection algorithms using remote sensing technology for estimating building damage.Most of them consider the detection of 2D changes to detect the building damage (Brunner et al., 2010;Thomas, 2010;Chesnel et al., 2008;Turker et al., 2005;Rathje et al., 2004;Chiroiu et al., 2002),several studies make use of the 3D information of scenes to detect the building damage (Liu et al.,2013;Dong et al, 2013;Rezaeian, 2012;Taneja et al., 2012;Turker, 2005;) hot issue in the fields of computer vision and photogrammetry, many great achievements have been done to improve the accuracy of geometry in multi-view 3D reconstruction, which is better to meet the requirements of practical applications (Furukawa et al.,2010;Frahm et al.,2010;Agarwal et al.,2009).The key idea of 3D building damage detection is the 3D Change Detection using 3D point cloud obtained from aerial images through Structure from motion (SFM) techniques.Based on this, a novel method is proposed to detect the building damage.The method proposed by this paper not only consider the height of the buildings, but also taking into consideration of the image's shape and texture.Therefore, this method fully combines the 2D and 3D information of the real world to detect the building damage.
The rest of this paper is organized as follows.Section 2 describes the proposed method.Section 3 analyses and compares the experimental results, followed by the conclusion in Section 4.

Methodology
The framework of proposed method is shown in Where pixel(i) is gray value of each point in the 3D cloud, z(i) is the height value of each point, z(min) is the minimum height value and z(min) is the maximum height value.In order to make 3D cloud transfer real world coordinate, this paper transfer the local coordinate system into the world coordinate system by using the POS information of the UAV images.The transformation matrix consists of a translation component, a rotation component and a scale factor.

Building vector extraction
In order to obtain the position of the building, this paper extracts the vector of building from the pre-disaster's 3D point cloud.The methodology is described as follows.Firstly, Noneground point is removed using morphological filter.Secondly, the building is extracted based on the region grow method.
Thirdly, the building contour is projected to the ground for obtaining the building vector.At last, each building is numbered according to vector information.

DSM Co-registration
Because of the same real world coordinates in the pre-disaster data and the post-disaster data, a further co-registration between the two resulting DSM is designed and performed to remove any shift in three dimensions between the two DSMs (X.Dai, 1998).

Change Detection
By utilizing the differences based on change detection method, DSMs acquired in 2.2 are subtracted to produce a difference image which represents the height changes between the two dates.After binarization and Canny edge detection of the difference image, we obtain the building damage suspected area.But height information change can't prove the building damage, building damage judgment also need the image's characteristics information.

Building segmentation and extraction
After obtaining the building damage suspected area, the suspected area were projected to the mosaic image in the pre-disaster and post-disaster, which obtains the building damage suspected area of the UAV images.
Then the building damage suspected area by the building vector guiding in 2.2 obtain the building of the UAV images in the pre-disaster and post-disaster.
There the detail descriptions are as follows.
(1) To mosaic the UAV images based on the characteristic method.Details process are as follows. a) The (2) Height change area project to the mosaic image, which obtains the suspect area.Set the project matrix 4 3× P was the internal and external azimuth elements of single view geometry, (2) where P is the project matrix 4 3× P , f is the camera's focal length, x 0 is the eccentricity in the x-coordinate direction, y 0 is the eccentricity in the y-coordinate direction ;X s , Y s , Z s are the coordinate of camera's centre in the world coordinate system, R T is a 3 × 3 rotation matrix.
Projection formula is: (3) After the suspect area project the mosaic image of pre-disaster and post-disaster, Segmentation building by vector guiding in 2.2 are obtained in the suspect area of the pre-disaster and post-disaster image.

Extract the characteristics evidence of the building damage
According to the result of building segmentation based on the building vector guiding in the pre-disaster and post-disaster imagery, this paper extracts the area and texture characteristics of building to determine the building damage.The extraction process are as follows.
(2) The Gray Level Coocurrence Maxtrix(GLCM) method is a way of extracting the second order statistical texture feature.This paper describe the texture characteristics by using the energy characteristics ASM of GLCM, the ASM value of the complete building is bigger, but the ASM

Access building damage
The characteristics of a damaged building are strongly dependent on many factors, such height change, area change and texture change.So only using a certain characteristics to determine the damage building is not accurate.According to the principles of evidence theory, this paper will give evidence to the various features of buildings damaged confidence in the probability distribution by the way of statistical experts, on the basis of evidence we build the theory model to assess whether buildings damaged or not through using various confidence coefficient.Building damaged confidence formula is as follows: Where R j is the membership degree of the evidence j, j ρ is the prediction setting confidence degree of the evidence j. n is the total number of evidence.
Finally calculate the building damage of confidence Judge P .

Experiments overall description
In

Results of comparison
In April 2014, author has investigated and measured on the Beichuan earthquake ruins.According to the local geographic data and on-the-spot investigation result, the number of building damage is 160.From the result , the number of building damage is 138(see Figure 5(c)).The part building damage can't be detected using the method proposed by this paper, mainly because this paper determine the building damage by using the building height, area and roof texture, but many building's wall happened to broke or collapse.Therefore, this paper's method can't judge this kinds of building damage.

Conclusions
This paper has presented a novel building damage detection method from high resolution UAV imagery.
Towards that end, we have proposed and tested techniques for image registration, building extraction, change detection and damage detection.This new one fully combines the 2D and 3D information of the real world to detect the building damage.The results, evaluated on real data, demonstrate that this method is feasible and effective in building damage detection.In the future we intend to use multiply view oblique UAV imagery and public-source imagery on the ground to detection the building damage, which is better to assess the level of the building damage.
.At present, detecting buildings damaged by using 3D feature of scene mainly makes use of the height information of Lidar data and 3D GIS model.But LIDAR data and 3D GIS are not only difficult to get The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-7, 2014 ISPRS Technical Commission VII Symposium, 29 September -2 October 2014, Istanbul, Turkey but also too expensive for many areas, especially the rural areas.With the rapid development of the UAV technology, aerial images of the UAV are easier to obtain in the pre-disaster and post-disaster period.In recent years, 3D reconstruction by images has been a

Figure. 1 .
Figure.1.Our work is divided into 4 parts.The first part is the DSM generation.The second part is the segmentation and extraction of the 3D building to obtain the vector of the building.In the following, while the height change information of damage buildings are extracted and compared from the pre-event and post-event 3D point cloud, 3D changes can be detected, namely, building damage suspected areas will be detected.After the image mosaics, building damage suspected areas are projected inversely mosaics images in the pre-disaster and post-disaster.Finally, after comparing the area and texture feature of building in the pre-disaster and post-disaster period with the vector of building, on the basis of evidence we build the theory model to assess whether buildings damaged or not through using various confidence coefficient.

Figure
Figure.1 Framework of the proposed method 2.1 DSM generated with Multiple Overlapping UAV Images Three-dimensional reconstruction from images is one of the fundamental problems in the fields of computer vision and photogrammetry, at present many multiview methods are capable of computing highly accurate 3-D point as well as camera calibration information simultaneously using the constraints imposed by feature matches across multiple images(so called "structure from motion").3D cloud are generated by using SfM algorithm based on the multiple overlapping UAV images, then DSMs are generated using 3D cloud.Each point's gray value of 3D cloud in the 256 colors gray images is computed by formula as follows:

The
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-7, 2014 ISPRS Technical Commission VII Symposium, 29 September -2 October 2014, Istanbul, Turkey extraction the feature of image based on the SIFT.b) The registration of the image: firstly, the rough registration of the image by using K-D tree to search the matching point; secondly, the accurate registration of the image by using RANSAC to delete the error point.Through the registration of image, the transformation matrix can ) of the building damage is 1; when Figure5(a)(b)) in the change suspect area.Finally, the damage buildings are detected by using the method of section 2.5 and 2.6.