INTEGRATION OF IMAGES AND LIDAR POINT CLOUDS FOR BUILDING FAÇ ADE TEXTURING

This paper proposes a model-based method for texture mapping using close-range images and Lidar point clouds. Lidar point clouds are used to aid occlusion detection. For occluded areas, we compensate the occlusion by different view-angle images. Considering the authenticity of façade with repeated patterns under different illumination conditions, a selection of optimum pattern is suggested. In the selection, both geometric shape and texture are analyzed. The grey level co-occurrence matrix analysis is applied for the selection of the optimal façades texture to generate of photorealistic building models. Experimental results show that the proposed method provides high fidelity textures in the generation of photorealistic building models. It is demonstrated that the proposed method is also practical in the selection of the optimal texture. * Corresponding author


INTRODUCTION
Regarding façade textures, building models can be divided into three categories: block, generic, and photorealistic models.Block models provide the geometric shape of buildings without façade texture.The difference between the generic model and photorealistic model is that the textures are acquired from a material gallery (Parish and Müller, 2001;Müller et al., 2006) and the real images, respectively.The photorealistic models are conducive to virtual reality because of their realistic and immersive appearances.Thus, texture mapping for building façades has become an important work for the generation of photorealistic building models.
Building textures may be derived from close-range images (Laycock and Day, 2006;Tsai and Lin, 2007) or aerial images (Frueh et al., 2004;Wu et al., 2007).Compared to aerial images, close-range images provide a wealth of façade information with high spatial resolution for texture mapping.However, the façade may be occluded by other objects, such as trees, cars, and so on, in image acquisition.Those occluded parts would cause the façade images unreal.In this reason, it's a major process to detect and compensate occlusion parts.Lidar point clouds with accurate 3D information provide a good way for occlusion detection.For occluded areas, one may compensate the occlusion by different view-angle images.
The façades of a building are usually with regularly distributed structure, window, terrace and sill for instance.Those repetitive structures form regular patterns in images.However, the repetitive structures could produce different appearances of image patterns due to different illumination, sun shadows, for instance.To make the appearance of the same pattern be consistent, we propose to use an optimum illuminated pattern for each repetitive structure on texture mapping.Thus, this paper proposes a model-based method for texture mapping using close-range images and Lidar point clouds.Considering the authenticity of façade with repeated patterns under different illumination conditions, a selection of optimum pattern is implemented to yield a realistic model.

METEODOLOGY
In this study, we combine three kinds of data, building model, Lidar point clouds, and close range images.Building model provides self-defined coordinate system with building façade information.Lidar point clouds are used to aid occlusion detection.Close range images come with spectral information and there texture characters are used to identify façades.The major works contain four parts: (1) data registration, (2) optimal image sorting, (3) occlusion detection and compensation, and (4) optimal texture selection and mapping.

Data Registration
In the first part, three data sets among building models, images, and Lidar point clouds, are co-registered to a working coordinate system.Based on the building models, threedimensional conformal transformation and bundle adjustment are employed to register Lidar point clouds and images, respectively.

Optimal Image Sorting
In this step, we analyze images with respect to spatial resolution for a working façade, and select a master image and set the priority for alternatives.For these images which cover a working facade, we compute the spatial resolution base on the length of façade's boundaries.The one with the best spatial resolution is chosen as the master image.The remaining images are sorted by the corresponding resolution.

Occlusion Detection and Compensation
The occlusion detection includes the detection of self-occlusion and foreground-occlusion areas.Self-occlusion is detected through Z-buffer algorithm.And we detect the foregroundocclusion in images by the comparison of building models and Lidar point clouds.We classify the LIDAR points in three categories: ground point, foreground-element point and building-structure point, as shown as Figure 1.Then, the collinearity condition equations are employed to back-project those foreground-element points onto each image.The areas of foreground-occlusion can be detected.At the last step, we compensate the occluded areas in the master image by alternative images in line with optimal image sorting.Thus, we obtain the master image with minimal occlusion.

Optimal Texture Selection and Mapping
For those building façades with repeated patterns under different illumination conditions, we select the optimal façade texture for the similar repeated patterns.In the selection, both geometric information and texture are analyzed.The geometric information contains the shape and size of the patch in object space, as shown as Figure 2. It's used to pick up the similar patterns initially.For each group of initial similar pattern, we analyze the grey level co-occurrence matrix (GLCM) (Haralick et al., 1973) and the intensity to determine final similar pattern groups and select the optimal façade texture.In reality, there may be additional objects on building facades, air conditioner for example.Those non-main objects may influence the optimal façade texture selection.Therefore, segmentation is employed in this study, and we only take the majority into account.We calculate the average value of GLCM analysis for each group.Considering the visual effects, the optimal façade texture is selected based on that the GLCM analysis near to the average value with high image brightness.At the last part, façade texture mapping is employed to generate the photorealistic models.

EXPERIMENTS
The test site locates in National Central University of Taiwan.The test model is with several repeated patterns, shown in Figure 3.The datasets contain eleven close-range images, as shown in Figure 4, and LIDAR point clouds.Information related to the test data is given in Table 1.Table 2 demonstrates the number of selected optimal textures in the experiment.In the geometric information classification, the façades are initially divided into seven categories.The second column in Table 2 indicates the number of patches for each category.After the GLCM and intensity analysis, we separate the initial group into two similar pattern sets in Category-1 and Category-2.Table 2.The results of optimal texture selection Figure 7~9 show the results of optimal texture selection via texture analysis for Category-1 to Category-3.According to the initial similar patterns in Figure 7 and

CONCLUSIONS
In this investigation, we have proposed a façade texture mapping method by integrating close-range images and LIDAR point clouds.In addition, both geometric information and texture analysis were employed to select the optimal façade texture for the similar repeated structures.Experimental results show that the proposed method provides high fidelity textures in the generation of photorealistic building models.It is demonstrated that the proposed method is also practical in the selection of the optimal texture.

Figure 1 .
Figure 1.The classification of Lidar point clouds classification

Figure 2 .
Figure 2. The illustration of initial similar patterns classification

Figure
Figure 3.The building model Figure 5.The master images Figure7~9show the results of optimal texture selection via texture analysis for Category-1 to Category-3.According to the initial similar patterns in Figure7and Figure8, it is observed that there are two types of similar patterns: pure brick wall and window structure.The texture distribution is also presented two groups significantly in the plots of Category-1 and Category-2.Few patches contain a car, because the acquisition time is different for close-range images and Lidar point clouds.But, it does not affect the final selection.There is only one type of similar pattern in Category-3.The distribution of texture plot is also closer than Category-1 and Category-2.