A method of 3D-GIS application aided with stereo panorama technology

: So far, three-dimensional geographic information system (3DGIS) has been developed and applied to many fields. However, data capturing is a challenge for complex environment modeling and visualization. Panorama is an elegant method for users to get a real view without being plagued by 3D measurement and modeling. Although someone claimed that panoramas can offer the function of measurement, the results are widely considered not reliable. 3D images composited by laser scanner data are another kind of precise data source to replace traditional 3D modeling. Laser scanners have obvious shortcomings, such as vast data and time-consuming processes. We propose a method of using stereo panorama technology to complement the three-dimensional data acquisition, processing and management. By using two cameras combined with MTI sensors, which provide pose and position measurement. We can obtain the stereo panorama at one time, which enables users to measure positions and orientations of particular features. The panorama can be uploaded to the 3D-GIS in a custom transmission format to provide stereo browsing of data capture sites. Combined with photogrammetry technology, users can also get accurate results by measuring interesting features on the panorama through interactive measurement interface. Based on Google Earth platform, we verified the effectiveness of our developed system. It stands out from traditional methods with its irreplaceable superiority in local 3D data acquisition and updating, as well as its direct data measurement.


INTRODUCTION
The purpose of 3D geographic information construction is mainly to provide users with a digital and life-like information space, in which users can analyze and process spatial information to get results of relative applications.
However, data are in large demand when constructing a 3D-GIS, especially when we need to build detailed 3D models for better visual effects.In general, this process of data gathering and model building requires a lot of human and material resources.Moreover, the data updating afterwards is more complicated than that of a traditional geographic information system.The complex work is not simplified even when it only implements simple functions like browse and query without complicated special analysis.
To solve this problem, we proposed a method of replacing the modeling, query and data updating processes in a traditional 3D-GIS by 3D panoramic models in complicated and important sites.This method takes advantages of real-time digital images and the method of photogrammetry.The method we proposed is superior to traditional ones in following aspects: 1) Data gathered are stored and managed in database directly without analysis, processing or complicated 3D modeling.It's meaningful to the current embarrassing situation of costly huge database building but few data usage.
2) Data updating is clear and concise.What we need to do is only data re-gathering with relevant equipment, regardless of complicated factors and processes in traditional spatial database updating.
3) This method is directly based on digital image data, so spatial vector data management and processing are reduced.Accordingly, the system is simplified, and the up-to-data state and the effectiveness of information updating are improved.
As mentioned above, the method we proposed is very promising, and it may become an important part of 3D-GIS in the future.This essay is structured as follows: we begin with a short description of relevant research in Section 2, and then come up with the theory and the measuring method of stereo panorama refactoring in Section 3. In section 4, we state the core content of this essay, the construction of a 3D-GIS integrated with stereo panoramas.

RELEVANT RESEARCH
It is widely studied over the years to introduce panorama images into 3D-GISs (or GISs), and there are many related commercial software systems, among which the panorama application of Google Earth is most popular.With this application, users can browse real sites, which to some degree are immersive [1].In China, there are also many so-called 3D panorama GISs, such as Roctec [2] and Mapscene [3].They claim that their systems have integrated panorama images into 3D-GIS, and users can not only browse panorama images continuously, but also do 3D object measurement.The main theory contains two aspects: 1) Match 3D point cloud obtained by laser scanners with images to get 3D images, as is stated in [11]; 2) Implement the measuring function by taking advantages of regular shapes on images and foregone constraint conditions.For the former, the aim of quick and real-time measurement is hard to achieve due to high-cost data acquisition and complicated data processing.For the later, measurement accuracy is not credible, so the results are for reference only.(Fangi, 2007(Fangi, ,2009(Fangi, ,2010)).In summary, the main purpose of these work is to realize the whole rebuilding of 3D object models through constructing panorama images.So it still costs a lot of resources.
Data accuracy is an important evaluation index for GIS applications, and in general rule, the precision should be without any doubt.So in this essay, we have a different purpose, to offer users not only good virtual presentation effects but also high-precision measuring results, so that we can meet professional users' demand.

MEASUREMENT OF STEREO PANORAMA
Based on stereo image pairs, users are able to attain 3D position of any object by stereo measurement.When this image pair is combined with real geo-reference, it can provide spatial coordinate of feature points, and reach the target of measure and analyzing objects with interactive operations.
The difference between stereo panorama and traditional photogrammetry is that every feature point of traditional photogrammetry has certain image frame and image center; but when it comes to stereo panorama, the progress of image stitching will make some image frame information lost, so it's impossible to measure according the original images' frame and center.There are two ways to make panorama: one is the camera has a fixed position and turn 360° to take photos(Figure 4A); another way is the camera moves along the focal plane (most used in street view images, Figure 4B).
It's obvious that the first way is not suitable to build stereo panorama, and the second way also has some shortages, such as camera mush move along a straight line, the interval must be equally divided .etc.When we want to get the positions of targets far from the station on the image high-precision results would be carried out after we set camera shooting routes properly and take pictures at suitable intervals by using 4.b method.In this article, we use the way showed in The way showed in figure 5.b is similar to that in figure4b.
We must consider relative offset and rotation between adjacent image pairs.There will be more fault if only consider rotation like building traditional panorama.
In order solve this problem, we built a specialized device, as is showed in figure 6, this device contains two same cameras, and can provide accurate distance between the two cameras measured by mm.An MTi motion tracker is used to attain the 3D pose and position of the device in real time.The angular accuracy of this motion tracker is less than 1° , positioning accuracy is less than 10m. Figure 6.A specialized device we built to build stereo panorama.
On the other hand, we can synchronize the two cameras in rotation and exposure.When doing measurements on the processed stereo panorama, we can ignore the rotation of baseline, then convert to unitized coordinate system.To make the problem simple, the original point of coordinate system is the center of baseline, x-axis is the same as north direction provided by MTi, y-axis and z-axis are decided by right-hand-thread rule.
Traditional photogrammetry method is used in the measurement of certain point, we can calculate 3D position of certain point using baseline's length and optical parallax.
Because this process takes a very short time, we can realize real-time measurement and spatial analyze.The detailed process is showed in Figure 7. Therefore, in this essay, we are not going to define 3D-GIS as   strict as that of tradition, but to call all software systems this name that express real environment, and provide users with information browse, query and analysis functions.Thus, in our definition, Google Earth is very good software of 3D-GIS.

3D GIS INTEGRATED WITH STEREO PANORAMAS
In addition, it is also the earliest non-professional GIS integrated with panoramas.
As is mentioned in Section 1, it is of great significance to replace complicated modeling by image sequences in depicting complex outdoor sites.The visualization effect of Google Earth panoramas is analogous to that of traditional 3D visualization methods in the aspect of browse, and there is no landscape distortion in any representing process.However after all, panoramas lack digital 3D information, so users cannot obtain spatial geometrical information, the main characteristic of GIS.In order to supply the gap, we introduce stereo panoramas into 3D-GIS.With the support of a background program, measuring operation of stereo panoramas is implemented, so that geographic information and spatial geometrical information of browsed objects are available when users are viewing panoramas.
In this essay, we regard Google Earth as a platform of        This contribution has been peer-reviewed.doi:10.5194/isprsarchives-XL-4-W3-123-2013

CONCLUSION
In terms of the spatial information system data acquisition is a kind of basic and decisive work.The acquisition of different types of data could influence the whole method of the following data processing, management and application.
Though there are many existing 3D GIS, part of which also declare that they can make measurement, provide limited measurement accuracy.The method provide in this paper would achieve accuracy which is the same with that in Terrestrial Photogrammetry theoretically.But the results showed in the experiment are not satisfactory for the limited accuracy of the camera calibration process as well as the MTI pose data.It could completely achieve the 3DGIS application requirement in the field of Surveying and mapping.
Except the real-time measurement of fine accuracy the method also enable users to do a quick data-update, sometimes on time update, and stereo measurement to aid in the real time and quick long-distance monitoring of the important targets when doing emergency rescues.It makes up the weakness that 3DGIS data is hard to update and express insufficient detail information of the real world.

The
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-4/W3, 2013 ISPRS/IGU/ICA Joint Workshop on Borderlands Modelling and Understanding for Global Sustainability 2013, 5 -6 December 2013, Beijing, China

Figure 1 .
Figure 1.Traditional data processing of a 3D geographic information system

Figure 4 .
Figure 4. Two different ways to make panorama.
of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-4/W3, 2013

figure 4 .
figure 4.b to make stereo panorama image sequence.
-GISs are generally considered as an extension of 2D-GISs in the third dimension.The relevant data model was once widely researched, but till now, there is no 3D-GIS in true sense.The main reason for it is the difficulty of setting up the 3D topological relation.In fact, the 3D topological relation is not in urgent demand in practical daily life, so many existing 3D-GISs, as well as some prevalent software like skyline, ArcGlobe, and EarthVision, mainly offer functions like browse, query and simple topology analysis.Although Google Earth has never claimed itself as a 3D-GIS, it has no essential difference from the specialty software listed above.Moreover, Google Earth has a larger group of users relevant to geography.Map World of China is another good example.

Figure 5 .
Figure 5.a.Rotate 360°at 2 different points.Optical parallaxes are different for different direction.

Figure 5 .
Figure 5.b.Two cameras rotate around a certain baseline's center.Optical parallaxes are changeless.
3D-GIS.Measurable stereo panoramas are linked to it by adding landmarks.Users can click interesting landmarks to activate the plug-in program for stereo panorama browse and measurement.It is in charge of the measurement and analysis operations.The method of integrating panoramas into Google Earth is detailed as follows: 1. Collect data of interesting spots outdoors with stereo panorama equipment.The data we need include: stereo panorama image sequences, 3D coordinates of the observation stations, orientations, and posture information of the image sequences; 2. The data we obtained are mainly images, so we choose the management style of file systems.For example, we can use Google file search engine to query original information kml files automatically with 3D positions and orientations we get in step 1 after brief arrangement; 5. Start the 3D-GIS (Google Earth in this essay, and other 3D GISs should implement the data-reading interface and the display module).The system reads in the information of stereo panorama image sequences, and then display them in the way of panorama landmarks on the visual interface; 6. Click the interesting panorama landmarks and their information is displayed in detail, such as: the images of shooting sites, shooting time, shooting-related personnel, names and related information of important land features (these data should be recorded in step 1); 7. Activate the plug-in program of stereo panorama measurement through the landmark linkage if users need more 3D information.Stereo browse and geometrical information measurement and analysis are available on the plug-in program interface; 8.Return to the 3D geographic information system after of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-4/W3, 2013 ISPRS/IGU/ICA Joint Workshop on Borderlands Modelling and Understanding for Global Sustainability 2013, 5 -6 December 2013, Beijing, China This contribution has been peer-reviewed.doi:10.5194/isprsarchives-XL-4-W3-123-2013all the related work.Google Earth has integrated many streetscapes (stereo panoramas) and implemented good browse functions, but these stereo panoramas lack immersion senses, as well as spatial measuring functions.The wok in the essay can well supply this gap.
In order to validate the practicability and operability of the stereo panorama integrated 3D GIS we carried out an experiment along the north shore of the Qinhe River with the device we have made to acquire the stereo image sequences.a) Stereo image acquisition Before data acquisition, we must do the preparation work of choosing the shooting route according to the image requirement.The shooting route should be as flat as possible to decrease the bend of the image sequence.To keep the brightness of the left image and the right in the same level the two cameras are adjusted properly to the same exposure settings.Also they are fixed stably at the same horizontal plane to avoid the vertical parallax in the synthetic stereo panorama.When taking pictures, the camera optical axes of the cameras in consecutive positions are kept parallel as far as possible.The fixed MTI on the camera holder is synchronous with the cameras and keep parallel to the optical axis of the two cameras so that more accurate posture data would be got to make a closer direction recovery (Figure 8).The first step is to take calibration images to carry out the intrinsic parameters of the two shooting cameras.Then we take images with 30 degree interval in all directions to collect a panorama image sequence of each fixed position.At the same time 3D pose data and position data are also saved in the course of taking images.Along the Qinhe River consecutive positions with even intervals are pointed out and the baseline between each two positions makes great influence to the following calculation.In order to realize an automatic homonym image pair search in the panorama image sequences, we use a telescopic range finder to estimate the distance from the targets to the camera in advance.

Figure 8 .
Figure 8. Data process interface when outdoor shooting

Figure 9 .
Figure 9.A panorama (right upper) and a synthesized stereo (right below)

Figure 10 .
Figure 10.Placemarks display and edit in stereo integrated 3DGIS

Figure 11 .
Figure 11.Targets distribution of the stereo measurement test of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-4/W3, 2013 ISPRS/IGU/ICA Joint Workshop on Borderlands Modelling and Understanding for Global Sustainability 2013, 5 -6 December 2013, Beijing, China

Table 1 .
Comparison between the distances measured by GE and our method (  : distance measured by GE,   :distance measured by our method)

Table 2 .
Comparison between the coordinates measured by GE and our method (  ,   : coordinates measured by GE,   ,   : coordinates measured by our method)