Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-3/W3, 543-551, 2015
http://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XL-3-W3/543/2015/
doi:10.5194/isprsarchives-XL-3-W3-543-2015
© Author(s) 2015. This work is distributed
under the Creative Commons Attribution 3.0 License.
 
20 Aug 2015
RASTER DATA PARTITIONING FOR SUPPORTING DISTRIBUTED GIS PROCESSING
B. Nguyen Thai1 and A. Olasz2 1Department of Cartography and Geoinformatics, Eötvös Loránd University (ELTE), 1/A Pázmány Pétersétány, Budapest, 1117 Hungary
2Department of Geoinformation, Institute of Geodesy, Cartography and Remote Sensing (F OMI), 5. Bosnyák sqr. Budapest,1149 Hungary
Keywords: GIS, GIS data decompositioning, Distributed processing, Geospatial data association, Linked dataset Abstract. In the geospatial sector big data concept also has already impact. Several studies facing originally computer science techniques applied in GIS processing of huge amount of geospatial data. In other research studies geospatial data is considered as it were always been big data (Lee and Kang, 2015). Nevertheless, we can prove data acquisition methods have been improved substantially not only the amount, but the resolution of raw data in spectral, spatial and temporal aspects as well. A significant portion of big data is geospatial data, and the size of such data is growing rapidly at least by 20% every year (Dasgupta, 2013). The produced increasing volume of raw data, in different format, representation and purpose the wealth of information derived from this data sets represents only valuable results. However, the computing capability and processing speed rather tackle with limitations, even if semi-automatic or automatic procedures are aimed on complex geospatial data (Krist´of et al., 2014). In late times, distributed computing has reached many interdisciplinary areas of computer science inclusive of remote sensing and geographic information processing approaches. Cloud computing even more requires appropriate processing algorithms to be distributed and handle geospatial big data. Map-Reduce programming model and distributed file systems have proven their capabilities to process non GIS big data. But sometimes it’s inconvenient or inefficient to rewrite existing algorithms to Map-Reduce programming model, also GIS data can not be partitioned as text-based data by line or by bytes. Hence, we would like to find an alternative solution for data partitioning, data distribution and execution of existing algorithms without rewriting or with only minor modifications. This paper focuses on technical overview of currently available distributed computing environments, as well as GIS data (raster data) partitioning, distribution and distributed processing of GIS algorithms. A proof of concept implementation have been made for raster data partitioning, distribution and processing. The first results on performance have been compared against commercial software ERDAS IMAGINE 2011 and 2014. Partitioning methods heavily depend on application areas, therefore we may consider data partitioning as a preprocessing step before applying processing services on data. As a proof of concept we have implemented a simple tile-based partitioning method splitting an image into smaller grids (NxM tiles) and comparing the processing time to existing methods by NDVI calculation. The concept is demonstrated using own development open source processing framework.
Conference paper (PDF, 1425 KB)


Citation: Nguyen Thai, B. and Olasz, A.: RASTER DATA PARTITIONING FOR SUPPORTING DISTRIBUTED GIS PROCESSING, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-3/W3, 543-551, doi:10.5194/isprsarchives-XL-3-W3-543-2015, 2015.

BibTeX EndNote Reference Manager XML