STUDY ON THE FEASIBILITY OF RGB SUBSTITUTE CIR FOR AUTOMATIC REMOVAL VEGETATION OCCLUSION BASED ON GROUND CLOSE-RANGE BUILDING IMAGES

Building 3D reconstruction based on ground remote sensing data (image, video and lidar) inevitably faces the problem that buildings are always occluded by vegetation, so how to automatically remove and repair vegetation occlusion is a very important preprocessing work for image understanding, compute vision and digital photogrammetry. In the traditional multispectral remote sensing which is achieved by aeronautics and space platforms, the Red and Near-infrared (NIR) bands, such as NDVI (Normalized Difference Vegetation Index), are useful to distinguish vegetation and clouds, amongst other targets. However, especially in the ground platform, CIR (Color Infra Red) is little utilized by compute vision and digital photogrammetry which usually only take true color RBG into account. Therefore whether CIR is necessary for vegetation segmentation or not has significance in that most of close-range cameras don’t contain such NIR band. Moreover, the CIE L*a*b color space, which transform from RGB, seems not of much interest by photogrammetrists despite its powerfulness in image classification and analysis. So, CIE (L, a, b) feature and support vector machine (SVM) is suggested for vegetation segmentation to substitute for CIR. Finally, experimental results of visual effect and automation are given. The conclusion is that it's feasible to remove and segment vegetation occlusion without NIR band. This work should pave the way for texture reconstruction and repair for future 3D reconstruction. * Corresponding author.


INTRODUCTION
Photo-realistic 3D models are nowadays required in many applications (Ortin and Remondino, 2005).Building 3Dreconstruction is an important part of work in Digital City and also is a significant part of street landscape, therefore how to carry out 3D visualization plays an important role in 3D city modelling (LI and Zhou, 2010).Terrestrial images for texture mapping streetscapes, which are captured along a narrow street, are hardly free of the occlusions which hinder the realistic façade texture.Thus, the full automation of texture occlusion removal has been regarded as a valuable research for rendering street scene (LIU and GUAN, 2010).Occlusions can be moving (e.g.pedestrians) or static (e.g.monuments) objects, which occlude the full and free visibility of the surface to be textured (Ortin and Remondino, 2005).This paper mainly discusses the static problem that building is occluded by vegetation.In the traditional multispectral remote sensing which is achieved by aeronautics and space platforms, the Red and Nearinfrared (NIR) bands, such as NDVI (Normalized Difference Vegetation Index), are useful to distinguish vegetation and clouds, amongst other targets.However, especially in the ground platform, NIR band is little utilized by compute vision and digital photogrammetry which usually only take visible light RBG bands into account.Therefore, in the ground closerange scene of buildings, whether NIR is necessary to recognize vegetation occlusion is discussed and tested.

Vegetation segment and removal of CIR image
Color infrared image (CIR) is not as popular as black and white for pictorial use, but it does have its place.Color infrared image is sometimes referred to as false color.This is because this image is designed to differentiate between colors rather than reproduce them accurately.And CIR is very important for data acquisition and updating, especially for vegetation (LYON et al., 1998).
The Normalized Difference Vegetation Index (NDVI) is a simple graphical indicator that can be used to analyze remote sensing measurements, typically but not necessarily from a space platform, and assess whether the target being observed contains live green vegetation or not.NDVI was one of the most successful of many attempts to simply and quickly identify vegetated areas and their "condition," and it remains the most well-known and used index to detect live green plant canopies in multispectral remote sensing data.Once the feasibility to detect vegetation had been demonstrated, users tended to also use the NDVI to quantify the photosynthetic capacity of plant canopies.The NDVI is calculated from these individual measurements as follows (NASA) : These spectral reflectances are themselves ratios of the reflected over the incoming radiation in each spectral band individually; hence they take on values between 0.0 and 1.0.By design, the NDVI itself thus varies between -1.0 and +1.0.It should be noted that NDVI is functionally, but not linearly, equivalent to the simple infrared/red ratio (NIR/VIS).
There are many methods that can be used for image segmentation.The NDVI is one of the most widely used indices for differentiating between vegetation and non-vegetation areas in remote sensing (ZHANG et al., 2006).For the NDVI, the threshold for vegetation extraction is usually positive and near to zero, it may vary from 0.05 to 0.15.Human supervision is helpful for selecting the best threshold from typical imagery.For our test data, non-vegetation areas are not well removed with a threshold of 0.0, while many areas are falsely removed with a threshold of 0.2.The best result is obtained with a threshold of 0.1.

Vegetation segment and removal of RGB image
In  The CIELAB color scale is an approximately uniform color scale.In a uniform color scale, the differences between points plotted in the color space correspond to visual differences Where (X, Y, Z) is the point to be converted, which can be obtained from Eq. ( 2).Let p represents X/X n , Y/Y n and Z/Z n respectively, then: Therefore, the classification of new patterns depends only on the sign of the expression w•x+b.After selecting samples, we can recognize two classes problem (vegetation or not) properly.

EXPERIMENTS AND RESULTS
The aim of this section is to evaluate the feasibility and effectiveness of the proposed occlusion detection technique.
The proposed method was implemented by C++.We can find that almost all the occlusion of vegetation is removed.

Conclusions and future work
An effective approach for extraction and segment the vegetation occlusion of building by RGB close-range imagery is presented.
The proposed image segmentation approach by CIE L*a*b and SVM has good potential for the removal of vegetation occlusion from imagery since it works on both CIR and standard RGB imagery.Especially, no matter whether vegetation is green or yellow, it adapts to the segment and removal of ground vegetation occlusion.We can draw the conclusion that RGB image can substitute for CIR image to recognize vegetation occlusion.
Tree that is a familiar occlusion has branches, leaves and trunk, which leads to spectrum difference, texture difference and geometry difference (edge).Hence, in order to get better result of occlusion removal, more information and feature (spectrum, texture and geometry) should be implemented and synthesized.This work should pave the way for texture reconstruction and repair for future 3D reconstruction.
the traditional multispectral remote sensing which is achieved by aeronautics and space platforms, the Red and Nearinfrared (NIR) bands.However, especially in the ground platform, NIR band is little utilized by compute vision and digital photogrammetry which usually only take RBG bands into account.Therefore, it is important to recognize vegetation occlusion in the ground close-range scene of buildings based on visible light RGB images.CIE L*a*b is very popular for image segmentation in computer vision.This is based directly on CIE XYZ (1931) and is another attempt to linearise the perceptibility of unit vector colour differences.Again, it is non-linear, and the conversions are still reversible.Colouring information is referred to the colour of the white point of the system, subscript n.The non-linear relationships for L* a* and b* are the same as for CIELUV and are intended to mimic the logarithmic response of the eye(Ford and Roberts, 1998).

Figure 1 .
Figure 1.Definition of CIE L*a*b , Y n , Z n ) is the tristimulus values for the illuminant(HunterLab, 2008), which is also called white point.Here illuminant can be: X n ,=0.312779,Y n =0.329184 , Z n =0.358037(ZHANG et al., 2006).Other parameters can be(Ford and Roberts, 1998;HunterLab, 2008):Cab =+ ±ΔL*means sample is lighter or darker than standard; ±Δa* means sample is redder or greener than standard; ±Δb* means sample is yellower or bluer than standard Many experiments have been carried out to compare the performance of image segmentation between NDVI and CIE L*a*b (ZHANG et al., 2006).So the CIE L*a*b approach is used for vegetation segment and removal.Another reason is that using CIE L*a*b, vegetation can also be extracted from visible light RGB images because the component a* is negative for vegetation in standard visible light RGB imagery and close to -120 for green vegetation.To segment RGB imagery with CIE L*a*b, a* from -0.15 to -0.05 should be applied as a threshold.The best threshold can again be obtained under human supervision.Meanwhile, it is of course not possible for NDVI to deal with RGB imagery.In order to get better removal result, a supervised learning method, support vector machine (SVM), is utilized for extracting vegetation.We choose (L, a, b) as characteristic variables.Given a set of training examples, each marked as belonging to one of two categories (vegetation or not), an SVM training algorithm builds a model that assigns new examples into one category or the other.An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible (BURGES, 1998).Linear SVM can be used in this paper.An n-dimensional pattern (object) x has n coordinates, x=(x 1 , x 2 , …, x n ), where each x i is a real number, x i ∈R for i = 1, 2, …, n.Each pattern x j belongs to a class y j ∈ {-1, +1}.Consider a training set T of m patterns together with their classes, T={(x 1 , y 1 ), ( x 2 , y 2 ), …, ( x m , y m )}.Consider a dot product space S, in which the patterns x are embedded, x 1 , x 2 , …, x m ∈S.Any hyperplane in the space S can be written as (BURGES, 1998) : of patterns is linearly separable if there exists at least one linear classifier defined by the pair (w, b) which correctly classifies all training patterns.This linear classifier is represented by the hyperplane H ( w • x + b =0) and defines a region for class +1 patterns (w•x+b>0) and another region for class -1 patterns (w•x+b<0).After training, the classifier is ready to predict the class membership for new patterns, different from those used in training.The class of a pattern x k is determined with the equation:

Figure 1 .
Figure 1.a) Standard false color composite satellite image.b) Result by NDVI.c) Result by CIE L*a*b.

Figure 2 .
Figure 2. a) True color close-range image.b) Result by CIE L*a*b with a threshold.c) Result by feature (L, a. b) with SVM.The first experimental data is satellite image that is composed by standard false color.And result of vegetation extraction by NDVI (with a threshold NDVI > 0.1) and CIE L*a*b (with a threshold a > 12) are assigned by green color in Fig 1. b)and c).We can find that there are almost the same results between these two methods.So the CIE L*a*b approach is used for vegetation

Figure 3 .
Figure 3. a) True color close-range image.b) Result by CIE L*a*b with a threshold.c) Result by feature (L, a, b) with SVM.The third experimental data is close-range building RGB image where there are yellow or brown vegetations.Under the threshold a<-12, there are not only many error segment areas in Fig 3. b), but also some withered and yellow vegetation isn't recognized by this method.Thus, SVM method based on selecting ( L, a, b ) as feature is implemented for removal vegetation occlusion.The experimental results are shown in Fig3 c).We can find that almost all the occlusion of vegetation is removed.
between the colors plotted.The CIELAB color space is organized in a cube form.The L* axis runs from top to bottom.The maximum for L* is 100, which represents a perfect reflecting diffuser.The minimum for L* is zero, which represents black.The a* and b* axes have no specific numerical limits.Positive a* is red.Negative a* is green.Positive b* is yellow.Negative b* is blue.Fig1 is a diagram representing the CIELAB color space (HunterLab, 2008).