EXTRACTION OF URBAN TREES FROM INTEGRATED AIRBORNE BASED DIGITAL IMAGE AND LIDAR POINT CLOUD DATASETS-INITIAL RESULTS

Timely and accurate acquisition of information on the condition and structural changes of urban trees serves as a tool for decision makers to better appreciate urban ecosystems and their numerous values which are critical to building up strategies for sustainable development. The conventional techniques used for extracting tree features include; ground surveying and interpretation of the aerial photography. However, these techniques are associated with some constraint, such as labour intensive field work, a lot of financial requirement, influences by weather condition and topographical covers which can be overcome by means of integrated airborne based LiDAR and very high resolution digital image datasets. This study presented a semi-automated approach for extracting urban trees from integrated airborne based LIDAR and multispectral digital image datasets over Istanbul city of Turkey. The above scheme includes detection and extraction of shadow free vegetation features based on spectral properties of digital images using shadow index and NDVI techniques and automated extraction of 3D information about vegetation features from the integrated processing of shadow free vegetation image and LiDAR point cloud datasets. The ability of the developed algorithms shows a promising result as an automated and cost effective approach to estimating and delineated 3D information of urban trees. The research also proved that integrated datasets is a suitable technology and a viable source of information for city managers to be used in urban trees management.


INTRODUCTION
Urban trees have many advantages such as preserving energy, improving water quality, minimizing greenhouse gasses and many other environmental pollutants, as well as connecting urban dwellers with nature (McPherson, E. G., 2006, Nowak D. J., 2007).In spite of efforts and capital spent on the conservation of trees, many city authorities often do not have an all-inclusive information on their condition (Yang, J., 2012).In order to realize numerous economic, environmental and sustainable decision-making processes, an accurate, up-to-date and indepth information on spatial distributions and health conditions of urban ecosystem is necessary.Accurate techniques for locating and mapping urban trees help city planners and other decision makers to better understand how much canopy cover exists, identify new planting, removal, or reforestation opportunities and what locations have the greatest need or potential to maximize benefits of return on investment.It can also help track trends or changes to the urban trees over time and inform future management decisions.Conventionally, this information is obtained through field surveying methods and interpretation of the aerial photography.Generally, ground surveying techniques are highly expensive, laborious (tedious), time-consuming and usually cannot be carry out over large areas.In addition, field surveying can only be carried out in areas reachable by the surveyors with insufficient or no data obtained in restricted properties and other unreachable areas.It is not easy, if not impossible to generate or create urban trees inventory for the whole city through field surveying.On the other hand, aerial photography does not directly provide 3D information of trees structure (Chen, et al., 2006) and is easily influenced by weather condition and topographical covers (Chen, et al., 2005).Therefore, somewhat insufficient information is obtainable about trees in many cities around the world, which is a major limitation for actualizing their benefits (Zhang, and Qiu, 2012).
Advancements in remote sensing tools have introduced laser technology which bridges the gap of satellite imagery inability to pass through the trees canopy.This permits dense and accurate measurements of underneath trees structure (Rahman, et al., 2015).Light Detection And Ranging (LiDAR) is an evolving technology which has the ability to generating a well-defined 3D representation of ground surface over wide spatial scales (Carter, et al., 2012, Reitberger, et al., 2009).The distance between LiDAR sensor and terrain features can be measured with a very high degree of accuracy by estimating the time taken by the laser pulse to travel from laser instrument and then return after being reflected from terrain feature (Persson, et al., 2004, Persson, et al., 2006).The capability of LiDAR to pass through vegetation has attracted remarkable concern from the field of natural resource management (Gaulton, et al., 2010, Hudak, et al., 2009, Liang, et al., 2007).From a forest management stand-point, LiDAR has been used to define information about trees (Coops et al., 2007, Brolly, et al., 2013, Lang, et al., 2006), measure carbon stocks (Patenaude et al., 2004), compute fuel quantity (Seielstad and Queen, 2003) and create habitat models (Vierling, et al., 2008), develop forest inventories (Zhang, C., 2010, Woods, et al., 2008).Even though considerable research has been carried out regarding LiDAR applications in forestry, its usage in the study of urban trees has been limited.As LIDAR applications in urban trees mapping expand, therefore, automated approach for tree detection technique is most likely to increase (Heinzel, et al., 2008).However, LIDAR systems have no band which makes it insufficient for vegetation classification, especially in urban forests with diverse species and high spatial heterogeneity.Digital multispectral imagine, usually possesses different bands, therefore, exhibit a great potential in identifying tree feature with their rich spectral contents.Airborne LIDAR data and imagery are highly complementary (Caldwell, et al., 2005), the images can validate the filtering accuracy (Jawak, et al., 2013) while the elevation information from LIDAR can be used to ortho-rectify images datasets (Flood, M., 2002, Savopol, et al., 2004).Highly dense LIDAR data with multiple returns per square meter would be overwhelming for tree crown depiction and for determination of crown shape while spectral properties can be used to differentiate tree objects (Holmgren, et al., 2008, MacFaden, et al., 2012).It is assumed that both data sources concurrently will be more successful for trees detection in contrast with any of them alone (Chen, et al., 2005, Zhang, C., 2010).
The objectives of this study includes extraction of shadow free vegetation features from the digital images using shadow index and NDVI techniques and automated extraction of 3D information about the vegetation features from the integrated processing of shadow free vegetation image and LiDAR point cloud datasets.The remaining parts of this paper describes, the study area and datasets used in section 2, the extraction techniques in section 3, the results and discussion in section 4 and finally, the conclusion in section 5.

Study Area
The study area is located in Besiktas district inside the city centre of Istanbul in north-western Turkey with a total area of 5,343 Km 2 (Başar, et al., 2011).Istanbul is among the most special cities in the world with its position as a bridge between Europe and Asia.The western part of the city is in Europe, and the eastern is in Asia.It is positioned between 28 0 01' and 29 0 55' eastern longitudes and 41 0 33' and 40 0 28' latitudes.Bosphorus strait (Figure 1) which connects the Sea of Marmara at the north and the Black Sea to its south divides the city into an Asian city closest to Europe and the closest European city to Asia (Gregory, T. E., 2010, Efe, et al., 2011).Istanbul is a typical urban area with complex spatial assemblages of vegetation, buildings, roads, and other man-made features.
Figure 1: Istanbul and the Bosphorus strait.

Datasets Used
Two datasets were used to achieve the objectives of this research study; namely:  Airborne based multispectral digital images which possess red, green, and blue bands (Figure 2) and near infrared band (Figure 3) at 0.1 and 0.5 spatial resolutions respectively. Airborne based LIDAR points cloud.
These datasets were collected in 2013 by BIMTAS Company in Istanbul, Turkey, using airborne laser scanning system.

Multispectral Digital Images (RGB and NIR
Bands) The multispectral images provide more details about spatial geometry and spectral information about surface of the study area used for detection and extraction of the vegetation features.The spatial reference of the images is WGS 1984, UTM Zone 35N.

Airborne LIDAR Point Cloud
The LIDAR data (Figure 4) provides an accurate, georeferenced and highly effective 3D spatial information about the shape and surface characteristics of the study area through x, y and z points commonly referred to as point clouds.It provides accurate height information which is missing in the digital images and also supporting information about crown shape (Hyyppä, et al., 2008).This information from LiDAR has the ability to orthorectify the digital images and generate a well-defined 3D representation of ground and features above the ground surface over wide spatial scales.The LIDAR data contains about 2,335,245 number of points.

THE EXTRACTION TECHNIQUES
Figure 5 present a work flow chart for identifying and extracting tree features from the integrated datasets.
Figure 5: The workflow chart.

Image Geometric Correction
The NIR image does not have the same spatial resolution and pixel depth with the multispectral image.Consequently, the NIR image which has 0.5 spatial resolution and 16bit pixel depth has been geo-rectified in order to have the same spatial reference with the RGB image which has 0.1 spatial resolution and 8bit pixel depth.
3.2 Shadow Index (Si) Shadow index is an indicator which describes presence of shadow objects for each pixel in a digital image.Consequently, in order to get rid of the confusing spectral problem between reflected spectra of specific kind of trees and the reflected spectra of the shadow of trees, the shadow values of the digital image have been determined using Equation 1 as below.SI = √ (256 -Red) (256 -NIR) (1) (Mustafa, et al., 2015).
where; NIR and Red are the Near Infrared and the Red reflectance bands respectively.
Furthermore, the shadow index image which provides precise shadows information was thresholded to detect absolute information.By utilizing this threshold value, a binary image was obtained with a value of 0 indicating the non-shadow objects and a value of 1 indicating the shadow objects.
Furthermore, the NDVI image which provides precise information about vegetation features was thresholded to detect absolute information.By utilizing this threshold value, a binary image was obtained with a value of 0 indicating non-vegetation features and a value of 1 indicating vegetation features.

Determining Shadow Free Vegetation Image
Shadow free vegetation image was determined by masking out features identified as shadows from the NDVI image.Hence, a binary image was created with a value of 0 indicating the non-vegetation features and a value of 1 indicating the shadow free vegetation features.

Extracting 3D Information of Vegetation Features
In order to achieve this objective, the shadows free vegetation image and the LIDAR datasets have been integrated to extract 3D information about the vegetation features.This task has been completely processed in a fully automated fashion using the Python programming tool.The modus operandi of the developed algorithm in processing this task includes: i.
Accessing and reading the shadows free vegetation image file.ii.
Detecting contour polygons of the vegetation features.iii.
Determining boundaries of each contour polygon of the vegetation features.iv.
Accessing and reading the LiDAR text file.v.
Integrating the shadows free vegetation image and the LiDAR data files.vi.
Extracting LiDAR points falling inside each polygon of the vegetation features.vii.
Saving the extracted LiDAR points into a new text file.
The mode of operation of this task have been represented using pseudo code flow chart.See Figure 6 below for more details.
Figure 6: Pseudo code flow chart of extracting 3D information about vegetation from the integrated datasets

RESULTS AND DISCUSSIONS
The purpose of this paper is to develop a feasible and efficient algorithm useful for extracting urban trees from the integrated airborne based LIDAR point cloud and multispectral digital imagine datasets.The above scheme includes extraction of shadow free vegetation features image from the digital image and automated integration of the LIDAR data and shadow free vegetation image in order extract 3D information about vegetation objects.

Shadow Index
It has been evidently proved that shadows presence is posing a great challenge during trees detection and extraction from the digital image (Mustafa, et al. 2015).This is due to the fact that NDVI normally fails to distinguish between the spectral reflectance of vegetation objects and that of their shadows.Therefore, the shadow index have been used to distinguished pixels which belong to shadow objects on the digital image from those of nonshadow objects by applying Equation 1to determines shadow values of the digital image on a per-pixel basis.
The result of shadow index (Figure 7) is a new image file with shadow values ranging from 1 to 239.The white pixels which have high shadow values represent the shadow objects while the black or dark grey pixels which have low shadow values represent the non-shadow objects.
By applying an absolute threshold value to the shadow index image, a binary image (Figure 8) was determined with a value of 0 indicating the non-shadow objects (i.e.black colour pixels) and value of 1 indicating the shadow objects (i.e.white colour pixels).The threshold value for shadow index image was estimated experimentally as 180.
Figure 8: Shadow index image after thresholding.

NDVI
The NDVI which is an index of plant greenness or photosynthetic activity has been used to distinguished pixels which belong to vegetation features on the digital image from those of other features by applying Equation 2to calculate NDVI values of the digital image on a perpixel basis.The output of this operation is a new image file (Figure 9) with NDVI values ranging from -1.0 to 0.989.The white pixels which have high NDVI values represent the vegetation features while the black or dark grey pixels which have low NDVI values represent the non-vegetation features.After applying an absolute threshold value to the NDVI image, a binary image (Figure 10) was created with a value of 0 indicating the non-vegetation features (i.e.black colour pixels) and value of 1 indicating the vegetation features (i.e.white colour pixels).The threshold value for NDVI image was estimated experimentally as 0.3.

Shadow Free Vegetation Image
The shadow free vegetation image was determined by masking out objects identified as shadows from the NDVI binary image.After the shadow objects were removed, the final output was turned into an image without shadow.Thus, a binary image (Figure 11) was created with a value of 0 indicating the non-vegetation features and value of 1 indicating the shadow free vegetation features.In this way, it became possible to get rid of the confusing spectral problem between reflected spectra of trees and that of their shadows.
Figure 11: Shadow free vegetation image.

Extracting 3D Information of Vegetation Features
The 3D information about the vegetation features (Figure 12) has been determined in a fully automated fashion by extracting LIDAR points belonging to vegetation features from the integrated processing of shadow free vegetation image and LiDAR point cloud datasets.
Figure 12: 3D information about vegetation features.

CONCLUSIONS
This paper presented workflow semi-automated approach for extracting urban trees from integrated airborne based LIDAR point cloud and multispectral digital image datasets.The paper proved that the integrated datasets is a suitable technology and a viable source of information for city managers to evaluate, enhance urban landscape patterns and gain a better understanding of the current spatial distributions, composition and extent of trees in an urban area.The ability of the developed algorithms shows a promising result as a fast and cost effective approach for estimating and delineated 3D information about urban tree, especially over a very large area.The extracted information will provides a snapshot of location, status and extent of trees in the study area which will be useful to city planners and other decision makers to better understand how much canopy cover exists, identify new planting, removal, or reforestation opportunities and what locations have the greatest need or potential to maximize benefits of return on investment.It can also help track trends or changes to the urban trees over time and inform future management decisions.
Finally, it is important to note that the object of interest in this study are the tree objects.However, critical analysis of the extracted LiDAR data (i.e.3D information about vegetation features) reveals that the extracted data consist of trees and other unwanted vegetation objects such as grassland and shrubs which would have to be completely filtered out from the dataset.Therefore, future work should concentrate on developing an approach or techniques which can be used to completely filter out these undesirable LiDAR points of non-tree features from the dataset.In addition, the designed algorithms have so far only been tested over the Istanbul urban area.Further research is needed in other urban areas with different species, forest compositions and/or spatial distributions in order to examine the robustness and extensibility of these techniques.