ROAD EXTRACTION AND VECTORIZATION FROM AERIAL IMAGE DATA

The objective of this study is the automatic extraction of the road network in a scene of the urban area from high resolution aerial image data. Our approach includes two stages aiming to solve two important issues respectively, i.e., an effective road extraction pipeline, and a precise vectorized road map. In the first stage, we proposed a so-called all element road model which describes a multiple-level structure of the basic road elements, i.e. intersection, central line, side lines, and road plane based on their spatial relations. An advanced road network extraction scheme was proposed to address the issues of tedious steps on segmentation, recognition and grouping, using the digital surface model (DSM) only. The key feature of the proposed approach was the cross validation of the road basic elements, which was applied all the way through the entire procedure of road extraction. In the second stage, the regularized road map was produced where center line and side lines subject to parallel and even layout rules. It gives more accurate and reliable map by utilizing both the height information of the DSM and the color information of the ortho image. Road surface was extracted from the image by region growing. Then, a regularized center line was modeled by linear regression using all the pixels of the road surface. The road width was estimated and two road side lines were modeled as the straight lines parallel with the center line. Finally, the road model was built up in terms of vectorized points and lines. The experimental results showed that the proposed approach performed satisfactorily in our test site.


INTRODUCTION
Automated creation and update of road maps from high resolution remote sensing data is one of the most important and challenging researches in the field of photogrammetry and remote sensing. Detailed and up-to-date road map information is necessary for a number of practical applications such as urban planning, transportation management, automotive navigation, disaster emergency response, and GIS related applications, etc. Road extraction from remote sensing data has been studied for many years (Grote et al., 2012;Lu et al., 2014;Ubukawa et al., 2014;Zhao et al., 2011). Although various automated methodologies have been developed in order to obtain more accurate road maps, quite a lot of challenges still remain due to the complexity of the environments in dense urban areas. Hence, there is a great demand for development of corresponding new strategies on automated road map creation.
For road extraction using remote sensing images, the methods used and the information that can be extracted vary greatly depending on the resolution of the images. When using low resolution (<10 m) image data, it is common to use edge extraction related methods to extract the road as linear information since it appears as a narrow linear object in the image (Doucette et al., 2001;Doucette et al., 2004;Wiedemann and Ebner, 2000). On the other hand, high-resolution aerial photographs and satellite images show the road in more detail and clarity. When using such image data, it is necessary to extract the road as an area with a width (Lu et al., 2009;Ravanbakhsh et al., 2008).
The extraction of road surfaces has been studied so far by using color information from high-resolution images. However, in * Corresponding author urban areas, there is a problem that many shadows cast by highrise buildings on the side of the road, which significantly changes the color tone of the road surface. In addition, various factors such as image acquisition conditions, trees, and moving objects affect the results of road area extraction. In addition to the color information of the image, a method that combines the height information of the DSM data with the color information of the image has also been studied by some researchers (Grote et al., 2012;Hu et al., 2004;Jung et al., 2006). DSM provides height data of each terrain cell, which is useful for direct classification, or segmentation of the off-terrain objects and the ground.
Given that there are so much good road extraction methods like what we introduced above, we would still like present this study that is novel on a complex vector road model as well as the according extraction and vectorizing method. First, we proposed a so-called all element road model which describes a multiplelevel structure of the basic road elements, i.e. intersection, central line, side lines, and road plane based on their spatial relations. An advanced road network extraction scheme was proposed to address the issues of tedious steps on segmentation, recognition and grouping, using the digital surface model (DSM) only. Next, the road map was produced in an accurate and reliable way by utilizing both the height information of the DSM and the color information of the ortho image. Road surface was extracted from the image by region growing. Then, a more precise center line was modeled by linear regression using all the pixels of the road surface. The road width was estimated and two road side lines were modeled as the straight lines parallel with the center line. Finally, the road model was built up in terms of vectorized points and lines. The experimental results showed that the proposed approach performed satisfactorily in our test site.

METHODOLOGY
The proposed road extraction approach includes two stages, which aim to solve two important issues respectively, i.e., an effective road extraction pipeline, and a high precision vectorized road map. The details are described as follows.

Extraction of road elements
In the first stage, we proposed a so-called all element road model which describes a multiple-level structure of the basic road elements, i.e. intersection, central line, side lines, and road plane based on their spatial relations. An advanced road network extraction scheme was proposed to address the issues of tedious steps on segmentation, recognition and grouping, using the digital surface model (DSM) only (Li et al., 2016). Figure 1 shows the flow chart of the proposed road extraction method. An overview of the analytical process is summarized below.

NDSM generation
NDSM is obtained from the difference between DSM and DTM. First, we use DSM data to generate DTM by applying an image morphological opening processing to remove ground objects such as buildings, trees, and mobile objects. Next, NDSM is generated by differential processing between DSM and the generated DTM.

Extraction of ground flat areas
Image morphological processing is applied to the generated NDSM to extract the so-called flat zones of a set of pixels with the same height and adjacent to each other. Flat zones are firstly extracted as the ground's basis. It refers to a connected region of the DSM, in which all the pixels have the same gray-level (i.e. height). Gray levelling and labelling are implemented to extract the flat zones. To avoid extracting rough objects, the approximate flat zone is introduced that use a parameter to filter the small area zones. A parameter is defined to permit the height difference within a zone. Ground flat areas are extracted by thresholding the height of the flat zones using the average height of the DSM to remove building areas. Some operations such as hole filling is implemented to the result and the ground is generated.

Extraction of street block envelopes
Street block refers to an area surrounded by streets, usually containing two rows of houses or several buildings. For these blocks, a Delaunay triangulation method is applied to obtain their envelopes as single pixel curves. A block's outline can be represented by an envelope, which should be close to its out extent and in a smooth shape. Here we choose the convex hull to represent the outline. The convex hull is the maximal alpha shape. The advantage of convex hull are as follows: it is unique and independent of any parameters because of the characteristics of Delaunay triangular; the lines of the hull are straight which can be used to create straight central lines in the following stages.

Extraction of candidate road center line
The center line in this stage is the contingent single pixel width image object. It is going to be used as the seeds of accurate road surface extraction in the next stage. Because of the complexity of the real world, the extraction of road center lines has to follow some cross validation process.
In order to extract the candidate road center line, a distance image of ground flat areas is generated by using distance transform method, where the distance is defined as the shortest one of a pixel in a region to its boundary. Therefore, a pixel which is farther away from the boundary have a larger value. The road center line consists of pixels with the local maximum values of the distance image. The generated distance image is used to extract the candidate road center line by a watershed image segmentation.
2.1.5 Determination of road center line, road side lines, and road intersections Road center lines, road side lines and road intersections are determined by a cross validation from each other. First, image morphological dilation was performed to the candidate road center lines and got the road polygons Pr, and the overlayed parts of the blocks' envelope curves with Pr were extracted as candidate road side lines. Next, we performed image morphological dilation again to the candidate road side lines and got the side line polygons Ps, and extracted the overlayed parts of the candidate road center line with Ps as the road center line. Here, the road center line is intermittent because the road intersections have not yet been extracted.
It is necessary to extract road intersections in order to connect intermittent road center lines. To extract a road intersection, first, a 5×5 template is moved along the candidate road center line. A candidate intersection is extracted if there are more than 7 pixels of the candidate road center line inside the template. Since some of the extracted candidate intersections are mis-extracted, the extracted road center line is used to remove road intersections that are not adjacent to the road center line. From the vertices of all extracted road center lines, an expansion is performed along the candidate road center lines. When it hits an intersection, the expansion is stopped. If the expansion does not hit the intersection even after expanding at a pre-defined distance, the expansion is stopped and the expanded part is also discarded. Finally, the road center line, road side lines, and road intersections are all determined.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 2 Flow chart of vectorized road map production

Production of vectorized road map
The objective of our study in the second stage was to produce the vector road map where center line and side line is regularized as straight line, with paralleled spatial relationship and parametered distance.
In the first stage, the basic road elements were obtained from the DSM of lower resolution and less accurate than the image. In this stage, the road surface can be extracted more accurately and reliably by utilizing both the height information of the DSM and the color information of the ortho image. For that, we a) generated seeds from the first stage's center lines, b) extracted road surfaces from the image by region growing, c) linear regressed a more precise center line accordingly using the pixels of a road surface, d) estimated the road width by the distances of the extremes of the road and created the side lines. Finally, the road model was built up in terms of vectorized points and lines. Figure 2 shows the flow chart of the vectorized road map production. The detailed stepwise implementation procedure is described as follows.

Generation of region growing seeds
The seed points were selected every certain interval from the road center line extracted in the first stage. We first partition the center line in a certain interval, ex. 10 pixels, then we label the regions of the center line sections as the seeds. Figure 3 shows the generated seed points.
Since the color and height of some road surface slightly changes in some places, the initial seed points for a road are expected to be singular and separate pixels instead of a whole line. The deviation of the color and height of the pixels on such a line would be greater than that of any neighborhood of a singular point which would cause a region grow to some unwanted pixels.

Generation of region growth control data
We were not employing region growing to segment the whole image into various semantic regions. The seeds were only of road surfaces, and the desired regions would be only of road regions. In case the regions grow to other types of land cover pixels, a controlling measurement has been introduced in the growing algorithm. The RGB image was converted to a grayscale image and edge extraction was performed using a Canny edge filter (Canny, 1986). In addition, the green index G/(R+B) is used to extract the vegetation areas. The edge pixels and the vegetation pixels were extracted from the image and taken as the barrier for the road surface. If a neighbor pixel is a barrier, the algorithm will not put it into possible growing neighbor.

Region growth of the road area
The road surface was grown using the hue information obtained by converting the RGB image into HSV color space, and the height information of the DSM data which expressed by equation (1). The thresholds for region growing were set to a hue of 0.5 and a height of 0.5 m. The result of region growth is shown in Here, refers to region j as a set of pixels. Refers to a pixel. ( ) refers to the neighborhood of region . refers to the edge pixel set.
refers to the vegetation pixel set.

Regularization of road center line
A morphological processing was applied to the road intersections extracted in the first stage, which were used to segment the grown road region into road sections. Then, the road center line for a section was modeled as a straight line and the associated parameters were calculated by linear regression taking all the pixels in that section into consideration. The large amounts of samples in linear regression will introduce a bit computation burden but will produce most reasonable center line without possible deviations by random sampling. For a dataset of huge extent, it is recommended to do random sampling to reduce the data size for regression.

Regularization of road side lines
Due to the influence of the canopies of the sidewalk trees, as well as the shadows of buildings and telephone poles, the complete road surface may not be extracted at some places when using the region growing method, which may result in inaccurate calculation of road width and road side lines. In order to solve this problem, the points farthest from the road center line among points on the road surface boundary were selected as extreme points. Figure 5 shows the extracted side boundary and the extreme points. Using the average distance of the extreme points to the center line, the road width was calculated, and finally two road side lines were modeled as the straight lines parallel with the center line, and evenly locate in its both sides in a distance of half road width. The regularized road central line and side lines are shown in Figure 6, respectively.

RESULTS
The study area is a densely built-up area in the Chiba, Japan. In this area, there are a variety of building types such as detached houses, apartments, tall condominiums, schools, factories, parks, ballfields, waterways, railways, and parking lots. The roads include slightly wider ones among large building blocks, and narrower ones among smaller building blocks, and elevated highways.
The aerial images are acquired by a multi-line digital airborne sensor ADS40. The orthoimages and DSMs data are generated by the Pixel Factory photogrammetric system. The resolution of DSMs is 0.5m and that of orthoimages is 0.2m. Figure 7 shows the road regularization result of the study area. It can be confirmed that the center line and side lines of the general roads have been correctly extracted across the whole study area. The elevated highways in the upper left of the study area have not been extracted. Since elevated roads, like buildings, have a height relative to the ground, their extraction needs to be studied in the future.

CONCLUSIONS
There is no doubting that automated creation and update of road maps is very important for urban planning, transportation management, automotive navigation, disaster emergency response, and GIS related applications. In this study, we present an automated road extraction and modeling method using aerial ortho image and DSM data of urban areas. The extracted road center line, side lines and intersections were encoded and vectorized during the procedure of road extraction. The experimental results showed that our approach performed satisfactorily in our test site. Our main contributions are as follows: We proposed a road network model including both line and area, represented by road central line, side lines, and intersections. They are self-contained and spatial relevant, and can be used in reasoning.
We presented a cross validation method for the road model, to remove the false intersections, false branches of the central lines.
We proposed a model to generate road area map. The road model can be built up in terms of vectorized points and lines.
In the future, we will apply the developed method to more areas to verify the extraction rate. Also, we will assess the accuracy of the road area extraction results in comparison with a real road map, and thereby improve the method for better performance.