INTEGRATION OF TREE DATABASE DERIVED FROM SATELLITE IMAGERY AND LIDAR POINT CLOUD DATA

Abstract. 3D tree database provides essential information of tree species abundance, spatial distribution and tree height for forest mapping, sustainable urban planning and 3D city modelling. Fusion of passive optical satellite imagery and active Lidar data can potentially be exploited for operational forest inventory. However, such fusion requires very high geometric accuracy for both data sets. This paper proposes an approach for 3D tree information extracted from passive and active data integrating into existing tree database by effectively using geometric information of satellite camera model and laser scanner scanning geometry. The paper also presents the individual methods for tree crown identification and delineation from satellite images and lidar point cloud data respectively, the geometric correction of tree position from tree top to tree base. The ground truth accuracy assessment for the tree extracted is also present.



INTRODUCTION
3D tree database provides essential information of tree species abundance, spatial distribution and tree height for forest mapping, sustainable urban planning and 3D city modelling.Tree database with important stand level attributes, such as tree location, species, diameter at breast height (DBH), tree height, and crown size can be integrated from multiple data sources such as field surveys, aerial imagery, passive and active remotely sensed data.With the improvements of spectral and spatial resolution in optical sensor technology, expensive and timeconsuming field surveys could be partially replaced by multispectral high-resolution imagery acquired by remote sensing satellites (Kamal et al. 2015, Kubo et al. 2005).Individual tree crowns can be delineated from high resolution satellite imagery (Song et al. 2010).Some structural parameters such as crown size, crown closure, and canopy structure can also be extracted.Tree growth can be evaluate over time.Spectral indices may also give information of tree health conditions.Besides optical satellite imagery, laser scanning and light detection and ranging (Lidar) technology (Zhen et al. 2016, Naesset 2007) has been broadly applied and has contributed to the efficiency of local-scale inventories.Airborne laser scanner (ALS) data (Koch 2006) provide high density 3D point clouds at high spatial resolution and vertical accuracies, enabling extraction of canopy height, above ground biomass estimation and individual species identification based on structural attributes.
While Lidar provides tree structural information, spectral data from satellite sensors provide information related to plant physiology and chemistry.The reflectance at various spectral bands and spectral indices constructed from these bands can potentially be used for monitoring tree stress and health conditions.The spectral attributes, together with structural information may aid in species differentiation and identification.Fusion of passive optical satellite imagery and active Lidar data can potentially be exploited for operational forest inventory.However, such fusion requires very high geometric accuracy for both data sets.Depending on the satellite viewing geometry, the * Corresponding author disparity between the tree top and the tree base of a tree on the satellite image varies according to the tree height.Thus, tree crowns in the georeferenced orthorectified image are displaced from their true geolocation.Due to its vertical scanning geometry, ALS data provide true locations of trees.Since different trees have different heights, it is a challenge to match tree crowns delineated in satellite imagery to the corresponding tree crowns extracted from ALS.
In this paper, we present In this paper, we present the methodology for automated individual tree crown extraction and delineation from high resolution satellite imagery and from ALS data, respectively; 3D geometry processing approach for geolocating tree crowns delineated from satellite imagery using combined information of satellite camera model and canopy height model (CHM) derived from ALS data; the integration of tree attributes from both sensors into a tree database; and the ground truth accuracy assessment of the results.

DATA SOURCES AND PREPROCESSING
The test site of this study is located at Ang Mo Kio town in Singapore, consisting of mixed forest, park, roadside trees and residential areas.

CHM data processing
ALS data of the area were collected at vertical and planimetric accuracy of 0.15 m or better with a density of at least 5 points per square meter.Raster images of CHM with 0.5 m per pixel resolution were derived from ALS data using a commercial off the shelf software.Small gaps within tree crowns in the CHM image are filled with neighbouring pixel values.The buildings and roads are cleaned according to the spatial characteristics.

Satellite imagery processing
High spatial resolution satellite images of the test site were collected by the WorldView-2 (WV2) satellite within several days from the acquisition period of the ALS data.The WV2 satellite images have 8 multispectral bands with 2 m per pixel resolution and a panchromatic band with 0.5 m resolution.The 8 spectral bands enable accurate classification of the different land cover types present at the test site.Besides individual spectral bands, several mathematical combinations of different spectral bands and a variety of spectral indices are also useful for the study of vegetation biophysical parameters.The Normalized Difference Vegetation Index (NDVI) constructed from the reflectance at the red and near-infrared (NIR) bands (Tucker 1979) has often been used for canopy mapping.Differentiation of bare land, grass, and bush from trees can be enhanced by other indices, such as the NIR to red band ratio as well as the ratio of the reflectance at the red-edge and red bands.
For better spectral analysis and comparisons of images acquired at different time period or by different sensors, the digital numbers of WV2 images should be converted to spectral reflectance values.The digital number is first converted to the top-of-atmosphere (TOA) spectral radiance (Todd et al. 2010) according to Eq. 1, where the subscript i denotes the band number,   is the TOA band-average spectral radiance,   is the absolute radiometric calibration factor,   is the pixel digital number, and ∆  is the effective bandwidth.The radiance is then converted to TOA spectral reflectance   by assuming a Lambertian reflecting target, where   is the band-averaged solar spectral irradiance for a given band, d is a correction factor for the different earth-sun distance at different days of the year and  is the solar zenith angle.Atmospheric correction is performed on the TOA reflectance by correcting for Rayleigh scattering and gaseous absorption.
Pan-sharpened images with resolution of 0.5 m/pixel for spectral reflectance of each band are produced using an in-house developed pan-sharpening algorithm that preserves both spectral fidelity of the multispectral bands and spatial resolution of the panchromatic band.

Existing tree database
The existing tree database with semantics of species, location, DBH, tree height and biomass was compiled by the National Parks Board from ground survey data.The database currently stores approximately 500,000 trees in parks and along the roads.The database is used for integration and also for accuracy assessment for the trees extracted.

METHODOLOGY
The CHM images are grey level raster images representing tree height while the WV2 satellite have 8 multispectral bands representing signal strength.Thus, different algorithms are used for extracting tree crowns from these data.

Tree crowns delineating from CHM
The CHM layer was segmented using an object-based multiresolution segmentation approach according to height homogeneity and local likelihood.Non-vegetation areas were removed according to the standard deviation of CHM and the shape (deviation from roundness of tree crowns) of the objects.Individual tree crowns were delineated using a watershed segmentation method (Beucher 1994) and the local maxima.Multiple thresholds were applied to watershed for overflow according to area, roundness, mean of CHM, standard deviation of CHM, etc.The tree crowns were smoothed by applying morphology filters together with region growing.Finally the tree crowns were reconstructed from CHM with attributes of crown size, centre position and height (Fig. 1).
Figure 1.Tree crowns extraction and delineation from CHM

Tree crowns delineating from satellite imagery
The satellite imagery of WorldView-2 was first orthorectified using a fine digital elevation model.The pixel digital numbers were then converted to spectral reflectance.The multispectral and panchromatic bands were fused to form pan-sharpened images.The normalized difference vegetation index (NDVI) as well as several other spectral indices were computed.
An object-based multiresolution segmentation approach was used to roughly eliminate non-tree areas first, then the tree class was extracted precisely with the aid of a few spectral indices.The individual tree crowns were delineated from the identified tree class using watershed segmentation.The tree crowns were then smoothed by morphological filtering and region growing.
Finally the spherical tree crowns were reconstructed.

Spectral indices generation from satellite imagery:
Spectral indices of ndvi, grvi (green-red vegetation index), ratio of red edge and ndvi (ratio_re_ndvi), ratio of nir and red (ratio_nir_red), average intensity of nir, red, green and blue (br4) and average intensity of red, green and blue (br3) for the satellite imagery were generated from the reflectance as in Eq. 4,

Tree segmentation and classification:
An object-based multi-resolution segmentation was applied to divide image layers of interests to relatively homogenous objects based on spectral and spatial/contextual properties.The neighbouring fragmented objects were then merged if the difference of their mean image layer reflectance values was small.The objects were classified into a set of classes such as cloud, building, shadow, agriculture, shrub, tree, bare land, water, etc., according to the mean image layer reflectance values and the indices such as ndvi, ratio_nir_red, br3, etc.
Tree class normally has high ndvi and low reflectance of blue.Compared to trees, bare land has higher ratio_re_ndvi and br4; buildup area has higher yellow, nir2, and lower value of ratio_nir_red, ndvi; grass has higher value of yellow and br3; and shadow has lower value in any indices.In practice, a relatively higher threshold for ndvi and a lower threshold for blue band are set to eliminate only non-tree areas first.In the rest of the areas, the remaining non-tree classes are eliminated gradually by other criteria.

Tree delineation:
In satellite images, tree crowns normally appear as spherical objects with light or dark patches between neighbouring trees.The watershed transformation (Beucher 1994) is able to separate objects from the background, as well as from each other.NDVI is a useful layer in separating trees as the tree tops usually have higher ndvi and edges have lower ndvi.Moreover, br3, green index and nir layers also contribute to the separation of trees by watershed transformation.
A morphology filter with opening and closing operation is applied to the individual tree outlines to smoothen the crowns of trees and fill small gaps.Region merging, border optimization and reshaping are also applied to the delineated tree crowns.

Integration of tree database
Tree crowns extracted from CHM and WV2 images cannot be integrated into tree database without geometric correction.The disparity between the tree top and the tree base of a tree on the satellite image varies according to the tree height and depends on the satellite viewing geometry.Tree crowns in a georeferenced orthorectified image are displaced from their true geolocation.In comparison, Lidar scanner used in this project has a much higher geometric accuracy.With the vertical scanning geometry, ALS data provide true locations of trees with centimetre-accuracy.
The camera model of WV2 satellite is a transformation that matches 3D map coordinates with image coordinates.If the height of a tree crown in the image is known, the crown can be shifted to its correct location above the tree base using the satellite camera model.As the crown height is yet unknown in the WV2 image, but every tree crown in the CHM image already has the height attribute, we instead transform the tree crowns of CHM from their base positions to their respective crown top positions on the orthorectified satellite image using the satellite camera model.

Worldview-2 camera model:
A simplified reverse camera model of WV2 is expressed in terms of a set of Rational polynomial coefficients (RPC).The RPC model expresses the normalized column and row values in an image, ( , ) as a ratio of polynomials of the normalized geodetic latitude, longitude, and height (, , ℎ).Normalized values are used instead of actual values in order to minimize numerical errors in the calculation.The scales and offset of each parameter are selected so that all normalized values fall in the range [-1, 1].Each polynomial is up to third order in (, , ℎ) having as many as 20 terms.The rational functions are (5) where __, __, __ and __ are 20-term vectors of coefficients that are given in a metadata file that comes together with WV2 products.
For a straight vertical tree with ground geolocation(, ,   ), and tree height of Δh, the corresponding crown top is (  ,   ), and base is (  ,   ) in a satellite image.These coordinates are related by the RPC camera model, (, ,   + ℎ)  → (  ,   ) Thus, if the height of a tree is known, its crown can be shifted to its correct location ( ′ ,  ′ ) using the satellite camera model.

Integration of crowns of CHM and WV:
As the crown height is yet unknown in the WV2 image, but every tree crown in the CHM image already has the height attribute, we instead transform the tree crowns of CHM from their base positions to their respective crown top positions on the orthorectified image.
The transformed crown tops of CHM should match the crown delineated in WV2 images.As the outlines of tree crowns delineated from WV2 and CHM images are not always the same, matching criteria are required.The crowns from WV2 and CHM are considered to be well matched and thus belonging to the same tree if the overlapping area is at least 80% of either crown area (Fig. 2a).The WV2 tree crown then takes the height of the matching CHM crown.In the case where two crowns not well matched, WV2 tree crown takes the height (fig.2b) or the weighted heights (fig.2c) of CHM The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W10, 2018 13th 3D GeoInfo Conference, 1-2 October 2018, Delft, The Netherlands according to the extent of the common area.With the derived crown heights, the tree crowns in the WV2 image are transformed back to their respective true geolocation using the satellite camera model.Finally, the tree attributes extracted from both CHM and WV2 data are integrated into the existing tree database.The tree database is expanded by integrating trees extracted from CHM and WV2 but not present in the existing database.

RESULTS
An example of tree crowns delineation from CHM data using the method described in section 3.1 is shown in Figure 3 with outlines of crowns in light green.An example of tree crowns delineation from a WV2 satellite image using the method of section 3.2 is shown in Figure 4 with the outlines of crowns in red.The tree crowns extracted from CHM and satellite images overlaid together before geometric correction are shown in Figure 5.This figure demonstrates the height-dependent disparities between the crowns from the two data sources (red for WV and light green for CHM). Figure 6 shows the geometrically transformed tree crowns of CHM in the orthorectified satellite image.The crowns from the two data sources are very wellmatched.Finally, the locations of tree base in WV2 images can be derived using the camera model.The crowns from both CHM and WV2 images with attributes of tree centre, tree size, diameter, height can be integrated into the existing tree database to enhance the attributes included in the database.Accuracy assessments is done by visual inspection for the crowns extracted from satellite image.In the validation area (Table 1), there are 844 detected trees.Among them, 51 trees are falsely detected while 22 trees are undetected.Overall, the omission error (ratio of number of undetected trees and actual trees) is 3%, the commission error (ratio of number of falsely detected trees and total detected trees) is 6%.Hence, the producer's accuracy is 97% and the user's accuracy is 94%.The objects which are falsely detected as trees are mostly small objects less than 100 m 2 in area or grass patches.The accuracy could be improved by improving the classification of grass from trees.
Accuracy assessment has also been done for the extracted crowns from CHM by comparison with the tree database (Figure 7).In the same validation area, there are 866 tree crowns extracted from CHM and only 182 trees are in the tree database.The discrepancy is due to the incomplete coverage of the tree database.The number of extracted trees is comparable to that extracted from the WV2 image.Among the trees recorded in the database, 174 trees are detected in CHM and 8 trees are undetected giving a producer's accuracy of 96%.

CONCLUSIONS
The automated methodology of individual tree crown detection and delineation from CHM and Worldview-2 imagery using an object-based multiresolution, hierarchical-level approach with watershed transformation is presented in this paper.Using the satellite camera model, tree crowns from CHM are geometric transformed to the orthorectified satellite image to match the crowns extracted from the image.Thus, the height of tree crowns in the satellite image can be assigned using the known heights from CHM.With the derived height, the base location for crowns in the satellite image can be derived using the satellite camera model.The results from the test site show that tree crowns from CHM can be made to match those from WV2 images after applying the geometric transformation.Accuracy assessment for extracted tree crowns is performed with ground truth data and visual examination.An overall accuracy around 96% is achieved.The geometric, structural and spectral attributes of tree crowns from both CHM and satellite data can be integrated into the existing tree database by updating the existing records or expanding new ones.

Figure 2 .
Figure 2. (a) Crown of CHM and WV matches well with common area (in magenta) > 80% ; (b) Crown of CHM and WV not wellmatched; (c) multiple crowns of CHM match one crown of WV.

Figure 3 .
Figure 3. Crowns (in light green) extracted from CHM (height in grey, the brighter the higher).

Figure 4 .
Figure 4. Crowns extracted from WV2 satellite image with outlines in red.

Figure 5 .
Figure 5. Crowns extracted from CHM (in light green) overlaid with crowns extracted from WV2 image (in red).

Figure 6 .
Figure 6.Crowns extracted from CHM (in yellow) geometrically transformed to the orthorectified WV2 image and overlaid with crowns extracted from WV2 image (in red).

Figure 7 .
Figure 7. Crowns extracted from CHM (in light blue) overlaid with centres of trees (in red) from database.