FEASIBILITY OF SMARTPHONE BASED PHOTOGRAMMETRIC POINT CLOUDS FOR THE GENERATION OF ACCESSIBILITY MAPS

: Accessible cities with accessible services are an old claim of people with reduced mobility. But this demand is still far away of becoming a reality as lot of work is required to be done yet. First step towards accessible cities is to know about real situation of the cities and its pavement infrastructure. Detailed maps or databases on street slopes, access to sidewalks, mobility in public parks and gardens, etc. are required. In this paper, we propose to use smartphone based photogrammetric point clouds, as a starting point to create accessible maps or databases. This paper analyses the performance of these point clouds and the complexity of the image acquisition procedure required to obtain them. The paper proves, through two test cases, that smartphone technology is an economical and feasible solution to get the required information, which is quite often seek by city planners to generate accessible maps. The proposed approach paves the way to generate, in a near term, accessibility maps through the use of point clouds derived from crowdsourced smartphone imagery.


INTRODUCTION
The automatic or semi-automatic generation of thematic maps from 3D point clouds derived from low-cost sensors' data is a topic of interest among research community. That is due to the growing demand from local authorities and collaborating agencies to exploit the available data and to generate new data at moderate cost, in the context of smart cities, to improve the life of the citizens. The generation of accessibility maps to facilitate the displacements of persons with reduced mobility (PRM) is an example of use of these data sets. An accessibility map is a thematic map that helps the mobility of disabled people tagging critical road and infrastructure elements. For example, in Figure 1, a screenshot of the City of Sydney's Accessibility map (City of Sydney, 2018) is shown. Here, they allow the user to check for the presence of stairs, moderate or steep inclinations, or accessible wharves among others. Figure 2 shows a screenshot of the City of Seattle's accessibility map (AccessMap, 2016). Here, the developers present directly the value of inclination of the streets and the user can decide if the street is accessible or not according to their disability.   Figure 3) are generated based on visual inspection and/or manual measurements of some critical road elements. Instead, we envision to make use of point clouds derived from smartphone images as an intermediate tool to speed up and automatize the generation of these maps. Point clouds are valuable tool to easily/quickly obtain the dimensions (length, height, slope) of sidewalks, steps and ramps either manually or automatically.
After getting these measurements from the point cloud, the structural or road element can be tagged according to accessibility criteria such as adequate (green), moderate (orange) and difficult/not suitable (red). Based on this information, several maps can be built and also other accessibility routing path can be generated (Biere, 2017 andLópez-Pazos, 2017). Dense and accurate point clouds have been mainly generated using laser scanning systems and more recently LiDAR based mobile and backpack mapping systems are being used for that purpose (Nocerino, 2017b). These solutions offer high quality point clouds but at an expensive cost. Consequently, the updating of point clouds is an expensive process. But, with the improvement of RGB cameras performance during the last few years (Nocerino, 2017, Masiero, 2106, the use of imagederived point clouds has become a suitable alternative. The use of these point clouds has the advantage of dramatically reducing the cost of point cloud updating without losing density and accuracy ( Figure 4). On the other hand, thanks to recent advances in Structure-from-Motion (SfM) SW pipelines and point cloud processing algorithms, the proposed mapping approach is suitable to be fully automatized. This overall strategy allows to expand the coverage and increase the updating of these maps using a collaborative mapping or crowd-sourcing approach where citizens share images from their smartphones.
The SfM approach (Schönberger and Frahm, 2016) is an efficient and robust technique for obtaining 3D models or point clouds using smartphone camera imagery (Nocerino, 2017), (Golodetz, 2018). The absolute scale and the proper georeferencing of the 3D models can be achieved using Ground Control, GNSS tagged images, or a combination of both. SfM is widely used, mostly because of the automation of the current photogrammetric software pipeline and lower costs especially when compared to the mobile and backpack based LiDAR point clouds. With the use of GPU-enabled modern processors, the time required to collect and process the data is significantly reduced (Remondino, 2017) which leads to processing times of about a few hours for large datasets to few minutes for datasets considered in this study.
Recent works of (Serna and Marcotegui, 2013) and (Rodríguez-Cuenca, 2016), based on mobile mapping laser point cloud, show the potential of fully automated point cloud processing for extracting urban curbs. (Balado et.al, 2017) proposed an approach to automatically detect structural elements of the buildings such as steps or ramps. In (Balado et al., 2018), an approach based on a combination of geometric and topological information is proposed to classify automatically urban ground elements in point clouds (sidewalks, roads, curbs and stairs).
In this paper, we focus on the generation of dense and accurate point cloud for various urban structural elements from smartphone photogrammetric imagery and then analyse their performance and suitability for accessibility maps generation. To do so the paper has been organized as follow. Firstly, the requirements needed to classify the set of urban structural elements according to an accessibility criteria are presented in Section 2. Then, in Section 3, the proposed methodology to acquire and process the images is presented. The experimental results are presented in Section 4. Section 5 presents some practical consideration as well as suitability of the proposed methodology. Finally, Section 6 summarizes the conclusions of the proposed approach and discusses future improvements

ACCESSIBILITY REQUIREMENTS
This section introduces the technical characteristics that must be taken into account to build new accessible structural objects/elements or to classify existing structures according to some accessibility criteria. In particular, we focus only on curb ramps, stairs and ramps. The technical characteristics vary depending on the country or region. For the study presented in this paper, the ones defined in the Catalonia and Spain regulations are considered: Catalonia accessibility code (Generalitat de Catalunya, 2010) and Spanish order VIV/561 (Gobierno de España, 2010). In Table 1 a brief summary of the technical characteristics of the elements of interest in this paper (curb ramps, ramps and stairs) are presented: • Curb ramps. A curb ramp is "Sloped area cut into curb" (Canadian Human Rights Commission, 2007). The main parameters that characterize curb ramps are the free pass width, the longitudinal and transversal slope and well as the ramp length (L).
• Ramps. A ramp is defined as "Any slope greater than 1:20 (5%)" (Canadian Human Rights Commission, 2007). For those structures, the main parameters are the usable pass width, the ramp length, the longitudinal and transversal slope.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy • Stairs or a series of steps or flights of steps for passing from one level to another; are characterized by the number of consecutive steps and the height and width of each of them.

Main technical characteristics
Catalan regulation

Consecutive steps
≤ 12 and no isolated step [3,12] Step width ≥ 30 cm ≥ 30 cm Step height ≤ 16 cm ≤ 16 cm and 54 cm ≤ 2H + W ≤ 70 cm According to requirements defined by accessibility specialists, these structure elements should be measured with centimetre accuracy and be less than 2 degrees of error. Thus, our target is to obtain an accurate and dense point cloud data enough to measure small elements with centimetre relative accuracy for linear measurements and an accuracy of less than 2 degrees for slopes. The global accuracy requirement is less strict and few meters is enough.

Data acquisition
In order to carry out the analysis, we have acquired multiple overlapped georeferenced images using two different mobile phones (Apple iPhone 4 and 7 series). The position of the pictures has been provided by the phones' GNSS receivers. In order to obtain adequate images, we have defined a data acquisition procedure allowing the accurate reconstruction of the point cloud.
• Curb ramps: two strips of overlapped images (~80%) taken surrounding the object of interest (curb ramp).
• Ramps: Sequence of photographs taken along the ramp combined with a set of photographs of the ramp from above and a set of photographs from the bottom where the complete ramp is viewed (overlap around 80%).
• Stairs: A straight line of overlapped (~80%) photos at two different distances or strips from the bottom and also a set of overlapped images from the top of stairs looking obliquely to the stairs.

Data processing
The acquired images have been processed with the Agisoft Photoscan software (Agisoft, 2018) to reconstruct point cloud data as shown in Figure 4. In the paper, we present the results of several processing approaches: 1) with and without geometric camera calibration parameters, 2) with different scale constraints and 3) procuring the images along single or multiple line of strips. In this way, we evaluate the impact of calibration parameters, scale constraints and image acquisition procedures over 3D reconstruction of point cloud.
The reconstructed point cloud is further processed to extract slope and height parameters of sidewalk and ramp. The slope is extracted by fitting a plane using RANSAC approach while height is measured using photogrammetric triangulation technique. The extraction of these parameters forms an important input towards the generation of accessibility maps. Taking this information as input, each structural element will be further classified (adequate, not adequate etc.) with a label following the accessibility criteria. We validate the extracted parameters using reference measurements taken over the pavement using ruler and inclinometer.

TEST CASES
In this section, the results of the feasibility analysis are presented. First, a short description of two selected test cases is done, followed by a detailed presentation of the main results of these tests.

Curb ramp:
As a first test case study, we have selected a single pavement structure that contains two elements of interest from the accessibility point of view, a curb ramp and a step ( Figure 5). The structure is located in an area with good GNSS constellation visibility. Figure 5. Curb ramp scenario.

Stairs and ramp:
As a second test case study, we have selected a structure with four stairs and a long ramp ( Figure 6). The structure is located in an area with poor GNSS constellation visibility.

Curb ramp
The curb ramp model was generated with the two smartphones: the iPhone 4S (iP4S) and the iPhone 7 (iP7). In order to adjust the scale, the measures of a square tile belonging to the rig were used. In this test, we checked the horizontal, and the vertical dimensions of the curb ramp as well as the dimensions and angles of some inclined elements. In Figure 7, one of the generated point cloud together with the elements of interest are presented.

Dimensions
First test intends to analyse the impact of using a pre-calibrated camera or to calibrate it during the adjustment. This test was done with full set of images acquired using iP7 camera. In Table 2 and  Table 3. Camera calibration parameters estimated using calibration field and self-calibrated using 1 and 2 strips.
Second test aims to see if it is possible to simplify data acquisition procedure and to use less images per target. The results of the test (Table 4) prove that even in this situation the performance of the system is good enough for accessibility mapping purposes. It can also be seen that the camera does not need a previous calibration to achieve the expected performance.  Table 4. Dimensions of curb ramp parameters of interest using 3D point clouds generated with reduced set of images from iPhone 7; pre-calibrated camera parameters vs self-calibration procedure.
This test was also carried out with an "old" camera (iP4) achieving the same conclusions (  Table 5. Dimensions of curb ramp parameters of interest using 3D point cloud generated with full set of images from iPhone 4S; pre-calibrated camera parameters vs self-calibration procedure.

Slope
Analogous to the test for validating dimensions, the test to check performance of slope measurements intends to analyse the impact of using pre-calibrated camera. In Table 6, it can be seen that for full set of images, there is no difference and that the results are within the requirements. After that, the test was repeated with reduced set of images, and, although the results get worse, they still fulfil the requirements (Table 7).  Table 7. Slope of first scenario target of interest using a 3D point cloud generated with reduced set of images and without taking into account the camera calibration.

Stairs and ramp
The stairs and ramp models were generated with just one smartphone: the iPhone 7 (iP7). In order to adjust the scale, the measures of a square tile belonging to the ramp were used. In this test, we checked the horizontal and vertical dimensions of the steps as well as the slope of the ramp. In Figure 8, one of the generated point clouds together with the elements of interest are presented.

Dimensions
Again, first test of this scenario intends to analyse the impact of using a pre-calibrated camera or calibration during the adjustment. This test was done with full set of images from iP7. In Table 8 Table 8. Dimensions of steps using 3D point cloud generated with full set of images from iPhone 7; pre-calibrated camera parameters vs self-calibration procedure.
Next step was to analyse the feasibility of using reduced set of images. It seems that, effectively, the approach is still valid but it is really on the edge as one of the value is above the acceptable threshold (  Table 9. Dimensions of steps using 3D point clouds generated with pre-calibrated iPhone 7; using full set of images vs reduced set of images.

Slope
Last test was devoted to compute the slope of the ramp which allows the entrance to the building. The test confirms that using a pre-calibrated camera and full set of images, the point cloud fulfils the use case requirements (Table 10).  Table 10. Slope of ramp using a 3D point cloud generated using a full set of images from pre-calibrated iP7 camera.

Test Cases final remark
To conclude this section, just to note that curb ramp technical characteristics are in compliance with the Catalan and Spanish regulations (Table 1), so the element can be tagged as adequate.
For the stairs and ramp case, the step height is slightly higher than the ones specified in the Catalan and Spanish regulations. The ramp slope is in compliance with the Catalan regulation but it is slightly higher than the Spanish upper limit. In addition, the usable pass width is not in compliance with the Catalan regulation. For this case, both ramp and stair could be tagged as moderate in terms of accessibility according to the Catalan and Spanish regulations.

SUITABILITY OF THE PROPOSED METHODOLOGY AND PRACTICAL CONSIDERATIONS
The previous results show that, from a technical and economical point of view, point clouds generated from smartphones are suitable for accessibility analysis. The solution is more economical than laser scanner not only because the camera sensor itself is cheaper, the size of point cloud is smaller but also from the point of view of the human resources required to collect data.
In order to collect enough data to map a city, the methodology allows a crowdsourced approach since no expertise is required to collect the images: pre-calibrated cameras are not necessary; in general, with a single strip is enough; and the main "difficulty" is to assure that a reference measurement is available. When working in city environment, it can be assumed that the pictured tiles will have standard measurements, and then, no manual measurements will be required. In case, the objects of interest are made of concrete, the photographer should be required to make some manual measurements.
Again, assuming a crowdsourcing approach for mapping a city, the 3D point cloud generation and analysis should be done on a devoted server or in the cloud, as suggested in (Nocerino, 2017) and (Golodetz, 2018). This is feasible since the amount of data a single user would upload is "small" enough for not threading the common data usage limit.

CONCLUSIONS
In this paper we have presented an approach to generate reliable and accurate point clouds for the generation of accessibility maps using smartphone camera images. By means of two test cases, we have proved that we are able to properly measure the main technical parameters of curb ramps, ramps and stairs. These results prove that, using this technology, the cost of generating accessibility databases can be drastically reduced without losing reliability on the results.