A HIGH-CURRENCY GEO-SPATIAL SERVICE GATEWAY FOR NATIONAL GEO- INFORMATION SERVICE PLATFORM

Today, more and more geospatial services are provided by the governments and enterprises to share various geographic information data and functions, and services-based application integration has become a trend. However, many problems existed in the geoplatform for Geographic information sharing while providing services in the form of API, such as the coexistence of different versions of the same service, similar service routes of different APIs, cluttered service protocols, and complex authority management, that makes the integration among different geographic information services difficult and reduces the development efficiency. There are already some API gateway technologies to solve the problem, but the characteristics of geospatial services are not considered in the existing product. To address these problems, this paper proposed a high-currency geospatial service gateway system for National Geo-Information Service Platform based on the opensource framework of Kong for realizing the unified management and authorized open. The system provides the whole lifecycle management and fine-grained control for the service, and the functions such as unified geospatial service access, protocol conversion, service management, authorization verification, rate limiting, and security protection are also equipped. The system has been released and integrated in the National Geo-Information Service Platform, supporting hundreds of millions of service invocation every day. The result proves it simplifies geospatial services management, deployment, and application, and benefits the exchanging and sharing of geographic information. * Corresponding author


INTRODUCTION
Web-based geographic information service is the important measure for public service of geographic information in the information age (Longley et al., 2005). More and more public governmental sectors and enterprises open their own geographic information data and functions to the developers in the form of APIs to realize the value-added use of geographic information data and technology, and improve the service level (Wu et al., 2015). In January 2011, National Geomatics Center of China released National Geo-Information Service Platform "Tianditu", which effectively promoted the sharing and application of the distributed geographic information resources around the country. By the end of 2020, more than 1000 APIs were provided and the number of registered developers reached 60,000. However, here are some problems such as the coexistence of different versions of the same service, similar service routes of different APIs, cluttered service protocols, and complex authority management (You et al., 2012). This results in an increase of the difficulty in services aggregation and reorganization during the development and application process. More important, the service bandwidth is usually occupied by some spider programmes, and it is different to identify the developer behind them, which has serious implications for normal service invocation (Du et al., 2010;Wang Yu and Ding Xie, 2017). Consequently, it is impendency to manage and control the open geographic information APIs and the invocation processes.
With the development of microservices, API gateway has become the standard component of an open platform, which can help to simplify APIs management, deployment, and application.
Further, a barrier can be established between internal and external systems, and form an application development ecosystem (Zhao et al., 2018). Currently, the main opensource API gateway frameworks are Kong, Zuul, Spring Cloud Gateway at home and abroad. Some research on API gateway has been conducted. TeraGrid GIScience Gateway was proposed to manages the complexity of the TeraGrid and provided a collaborative platform for GIS users to conduct geographical analysis (Wang and Liu, 2009). An HTTP based interface protocol adapter engine was introduced to simply the difficulty of web services invocation with various protocols (Jin Z.G, 2014). As to traffic limiting, an overload protection strategy based on URI configuration file in combination with gateway Zuul was designed to realize the traffic limiting function and guaranteed the stability and immediacy of the core service process (Li et al., 2019).
However, existing research work has paid more attention to the network protocol adapter, rate limiting algorithm and the microservices integration, but ignored some special features of geospatial services and application requirements, such as the conversion of map services and geospatial data, the developer console, better security control, high currency and cache process and user-based log analysis.
Aiming at the issues, this paper proposes a high-currency geospatial service gateway system for National Geo-Information Service Platform based on the open source framework of Kong. It has the following features: unified geospatial service access, protocol conversion, service management, authorization verification, rate limiting, and security protection, and etc.

KONG GATEWAY
Kong is an opensource, lightweight API Gateway that lets you secure, manage, and extend APIs and microservices. It is a Lua application running in Nginx and made possible by the luanginx-module. There are three components of Kong: (1) Kong server: it is a nginx based server to receive API request; (2) Cassandra/PostgreSQL: the operation data is stored in the database; (3) Kong dashboard: officially recommended UI management tools.
The functions of Kong can be customized through the plugins, and the plugin set is implemented in the lifecycle of the API request and response cycle. The multi-node cluster is also supported, this make Kong can handle more traffic and the network latency is guaranteed to be extremely low at the same time.
Although some features such as timeout handler, fallback strategy, data cache, API aggregation are missing, the extensive plugins and high performance make many famous enterprises like Samsung, Expedia, Yahoo! Japan and Nasdaq select Kong as the API infrastructure to integrate their business. The opensource framework of Kong provides a sound basis for building the geospatial service gateway. Some plugins can be developed to cope with the geospatial services and requirements and management.

ARCHITECTURE
The basic architecture of National Geo-Information Service Platform API gateway is as shown in Figure 1. It contains four ties: API resource tier (geospatial service API and geospatial data API), API gate way tier (core and admin), geospatial API market tier and API developer tier.

Map Services
Place Services  Figure 1. Architecture of National Geo-Information Service Platform API gateway Figure 2 shows the whole process flow of an API request. The system firstly verifies the legality of the URL, IP, and Token. If the cache data exists in the Redis and hits, the result will be returned directly. Otherwise, the URL will be redirected to the internal address and invoke the reverse proxy to the background programme. The new response data will be cached if the storage space is sufficient and the strategy is turned on.

Geospatial API resource
The geospatial API resources are divided into two groups: geospatial service APIs and geospatial data APIs. Geospatial service APIs refers to the map service (OGC WMTS/WMS, xyz tile Map, Restful Map), place service, location service, route service, geocoder service, geoprocessing service and etc. Geospatial data APIs consist of different open data from the government and enterprises, and the data format includes ESRI shapefile, geojson, and KML, which are also shared by service interfaces. During the process of service and data APIs exchanging and sharing, it is needed to unify, convert, handle the interfaces of different format form different platforms.

Geospatial API gateway
Geospatial API gateway consists of two models: API gateway core and API gateway administration. API gateway core is responsible for the physical access of APIs, and servers as a reverse proxy to hide real requests. During the process of API access, this model is used to implement access control, rate and flow limiting and safety management. All the APIs are open to the authorized users with tokens, and this can ensure the security of service portal. API gateway administration model is designed for the platform operational people to manage the platform conveniently. It is used to provide API management service, including API definition, approvement, release, management, auth assignments and safety strategy configuration, and log statistical analysis.

Geospatial API market
Geospatial API market is an open resource center for all users, which includes all service and data APIs registered in the system. Users can search the specified API by the name, category or provider, browse the service meta data and invocation requirements, and apply the access authorization.

Geospatial API portal for the developers
Geospatial API portal for the developers refers to management of the authorizations for different application scenarios. In order to distribute the infrastructure resource for the APIs more reasonably, three categories of developer are defined: individual developers, enterprise developers and government developers. The quota of the APIs invocation is different based on the developer category. The portal is also the entrance of the registration of the developers.

METHODOLOGY
In this session, the key technologies of the Geospatial service gateway for protocol conversion, security control, high currency, and log analysis are described in detail.

Multi-protocol conversion for geospatial Map services and data
Geospatial map services and data services are different from the traditional web services, which contains some unique properties. For instance, one map service may have several interfaces such as OGC WMTS and XYZ tiles, or the user needs specific response format of the data. This section describes how to deal with the situations in Kong by developing plugins (Figure 3).

Geospatial Map services conversion
： Generally, the geospatial services, especially the map services support more than one network protocol, such as HTTP/HTTPS and REST, and some services do not conform to the OGC specification. Multi-protocol conversion module provides the capability for the service access and processing based on the different protocols and specifications. The conversion templates have been configured in advance. After shielding the geospatial services with different protocols on the background, the user can access the standard and friendly services in the API market. For example, the developer invokes a restful OGC WMTS map service, but the original registered service only supports HTTP protocol. The whole protocol conversion process includes four steps: (1) service request parsing: parse the request URL and judge if the request type is the same as the provided service type; (2) service interceptor: send the request to the corresponding Servlet to process; (3) conversion controller: convert the map service based on the exclusive template; (4) construction and return: construct the processing results as the internal message for the further steps.

Geospatial data conversion ：
In the National Geo-Information Service Platform， there are many open geospatial data with various formats, such as ESRI shapefile, geojson, kml, and etc. Also, the formats of response data from many geospatial services like placename service, route planning are different. Therefore, the data service conversion module is responsible to transform the response data into the object that is descend in a certain language, and then convert it into the corresponding protocol plugin.  Figure 3． Multi-protocol conversion plugins for geospatial Map services and data

Security control
The geospatial services gateway is used to make the developers to access the APIs indirectly and safely, and protect the original services. Therefore, the security control is the key model for the system. It contains request URL verification, white/block list verification and Token verification.

Request URL verification:
The first step for security control is to check whether the request URL is legitimate, such as the parameters are complete, the length or character of Token is correct. The characteristics for different services' URLs have been stored in the Redis cluster. If the request URL is illegal and the request will be intercepted and an error message will be sent to the user.

White/block list verification:
When a request URL passed the verification, it will go to the next step to verify the source IP whether exists in the white or block list. If the IP exists in the white list, it will pass the verification and the service invocation quota is infinite. However, if the IP exists in the block list, the user cannot access any services even with the correct token. And the rest IP will be verified the access token.

Token verification:
When users access the geospatial services, tokens are needed. This module will check the validity of the token and authority in the request parameters. Only the user with the legitimate token and authority can access the service. Otherwise, the service cannot be invoked without authority even though the token is legitimate.

High currency process
There are more than 500 million geospatial services invocation everyday for National Geo-information Service Platform of China. The performance is critical to the system and time delays for must reach the millisecond level. The following technologies have been applied to ensure the high currency except proxy server and load balancing.

Coroutine based high currency:
In the scenario of high currency, the technology of multithread is often used to increase the performance for processing. The cost of multithread development is low, but the memory consumption is very high and the switching times are too many during different threads. In consequence, coroutine has been introduced to solve the problem. It can promote utilization rates of the thread based on the more flexible scheduling strategy and low task overhead.

Rate limiting:
By limiting the flow of the large traffic requests to block most of the requests, and only parts of the requests are allowed to enter the sever, which can prevent the system collapse effectively while facing with the high currency. The sliding window algorithm has been applied for rate limiting. It is a hybrid approach that combines the fixed window algorithm's low processing cost and the sliding log's improved boundary conditions.

Service fusion:
Service fusion is another protection strategy. when a service fails or large traffic requests leads to an excessive load for the system, the failed service should be cut and using an error message to replace the right response. The granularity of service fusion is accurate to the API level and the strategy includes the response time, response status, and the number of failed transactions.

Real-time log statistics and analysis
Since the situation of the system operating and the APIs invocation should be monitored in real time, a Flink-DWS of HUAWEI Cloud based dynamic flow computation framework was adopted to process the logs from the gateway. The Flink cluster in the system consists of three parts: data pulling (source), logical analysis core (transform) and output stream (sink). The consumer model was used in data pulling to fetch the data in the DIS channel and record checkpoint to ensure the integrity of the link. When the job is restarted abnormally, the data in the DIS channel will be read from the checkpoint automatically. After cleaning and transforming the data through the internal customed UDF, they will be written into multiple sink output streams for statistic and analysis. Logical analysis core mainly realizes log data cleaning, data formatting, log statistics, model calculation with services and users, space conversion calculation, user judgment and other tasks of the system, and part of the processed data is entered into CSS. The cache and persistence can be viewed in real time through Kibana. The other part of multi-dimensional statistical data is stored in the DWS database in different tables, waiting for the subsequent scheduled offline tasks to perform scheduled analysis.

IMPLEMENTATION
In order to verify the efficiency of the technologies and methodologies proposed in this paper, the geospatial API gateway system for National Geo-Information Service Platform "Tianditu" was developed and deployed on the HUAWEI cloud. As shown in Figure 4, the system was developed by the separation of front-end and backend, the front focused on the visual intersection and data presentation, and the background was responsible for the management process and implementing business logic. The relationship database of PostgreSQL is used to store the data of users, geospatial services, authorities and so on. For the high frequency usage data such as tokens, authorities, traffic quota, response cache were stored in the Redis cluster. As to the huge logs produced by the system, the GaussDB (DWS) of HUAWEI cloud was taken as the storage and analysis tool. Adopting message communication mechanism, modules of system can communicate timely and asynchronously to ensure the consistency of the data. All the severs were deployed by using the containers of Docker, the nodes can be increased or reduced easily and efficiently.  In Figure 6, the working situation of the system can also be controlled through a larger screen by the functions of statistic and analysis. All the data shown in the screen is calculated and The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B4-2021XXIV ISPRS Congress (2021 analysis through the log module of the system nearly in realtime. They can apply for the token to invoke geospatial services freely, and the token type is divided into browser, server, Android and IOS based on the application client. The service authority of each token can also be edit as needed, and the white list is provided to protect the token from being stolen.  Table 1, by comparing the original Kong gateway with the Kong-based geospatial service gateway in this paper, more advantages can be found for the geographic information application scenario. General functions have been supported in Kong, but the geospatial service gateway introduced has advantages such as supporting geospatial services, log analysis, monitoring and early warning. The GUI is more local and also very friendly for both the developers and administrators. Since January 2019, the system has been formally put into service. There are more than 1000 services have been registered and managed, and the data from the service monitor shows that it supports more than 40,000 geospatial API requests per second at the peak and forward delay timer was under 20 milliseconds. The usability of the system's core modules reached 99.95% and the invocation success rate exceeded 99.99%.

CONCLUSION
The main work of the paper is as follows: Firstly, the characteristics of the online geographic information service APIs has been analysed, and the general structure of function, technical framework and the modularization of system with the discuss of the functional and performance requirements. Secondly, to ensure the APIs can be managed unitedly and called safely, this paper designs the modules of authority authentication, policy management and traffic statistics based on the application Token, which is applied by the developer. When a new service is registered, some parameters such as access authority, access frequency, fusing threshold should be set. The whitelist mechanism based on requesting domain names or IPs is used to ensure the reliability of the source of the request and prevent Tokens from being stolen. Thirdly, the same geographic information service may have different access APIs, such as OGC WMTS and Restful interface. In order to improve the accuracy of traffic statistics and control the flow, a service transformation plug-in is developed, which unifies the various services to the default service in the backend. Finally, as a national public service platform, not only the services from the government should be provided via the system, but also those from the universities, research institutes, enterprises needed to be collected. The system supports the sharing geographic information services from the third-party to be managed and invoked with a unified development Token of the platform.
At present, the geospatial service gateway system has been released and integrated in the National Geo-Information Service Platform, supporting hundreds of millions of service invocation every day. While providing a unified gateway for registration services, it also provides the ability of basic data analysis for service publishers. Feature research in this system will emphasize: (1) further improving rate limiting based on the user access behaviors and (2) continually improving the usability and flexibility of the system.