Development of Open source-based automatic shooting and processing UAV imagery for Orthoimage Using Smart Camera UAV

Recently, aerial photography with unmanned aerial vehicle (UAV) system uses UAV and remote controls through connections of ground control system using bandwidth of about 430 MHz radio Frequency (RF) modem. However, as mentioned earlier, existing method of using RF modem has limitations in long distance communication. The Smart Camera equipments’s LTE (long-term evolution), Bluetooth, and Wi-Fi to implement UAV that uses developed UAV communication module system carried out the close aerial photogrammetry with the automatic shooting. Automatic shooting system is an image capturing device for the drones in the area’s that needs image capturing and software for loading a smart camera and managing it. This system is composed of automatic shooting using the sensor of smart camera and shooting catalog management which manages filmed images and information. Processing UAV imagery module used Open Drone Map. This study examined the feasibility of using the Smart Camera as the payload for a photogrammetric UAV system. The open soure tools used for generating Android, OpenCV (Open Computer Vision), RTKLIB, Open Drone Map.


INTRODUCTION
Recently, fast changing social developments, continued developing cities, a localized heavy rain from abnormal climate and other events are causing frequent changes in territories.Due to these changes, existing monitoring system becomes limited in time, space and costs, and we need real time monitoring not only for the periodic monitoring to prevent disaster and accidents but also to gain the most accurate and newest space information.Therefore, continuous real-time information data acquisition system to secure speed and the newest information is needed.(Choi and Kim, 2014;Park, 2014).As the drones (UAV, unmanned aerial vehicle) have become smaller and lighter, it became possible to use it in variety of categories such as military, disasters, accidents, broadcasting communication, and it is utilized according to its characteristics and purpose.Aerial photography using UAV has comparably low altitude to aircrafts or satellites and since it is capable of autonomic flight, it is economical, gets less influence from the climate, and able to acquire high space resolution data.Chiabrando et al. (2011) utilized UAV in obtaining and handling of low-altitude aerial photography to support antiquarian investigations through providing cadastral map.Siebert et al. ( 2014) used a rotor UAV to perform mobile 3D mapping for measurement of earth work project.There are many other continuing researches to use UAV as an analyzing system for photogrammetry.UAV loads observation sensor and receives manipulation and control remotely through link from the ground/ loading data of GCS (Ground Control System).In controlling UAV remotely, wireless communication system from UAV's serve system plays vital role.Wireless communication system sends data of UAV such as speed, altitude, and coordination to ground control system using MAVLink (Micro Air Vehicle Communication Protocol) communication protocol, it also connects UAV and * Corresponding author :cuchoi@pknu.ac.kr ground control system with RF modem, and plays important role in sending, receiving, or remotely controlling flight information such as 3D location and positions.In case of South Korea, communication using existing RF modem has communication range of 1-2km, and lines cross often.Thus this research puts its purpose on improving UAV's wireless communication system using existing RF modem with wireless communication technology like 3rd generation or 4th generation LTE, Bluetooth, WiFi of smart device and materializes communication system capable of data transferring and using UAV with communication system like it, implementing close aerial photogrammetry.

Telemetry module based on Android for UAV
Normally aerial photography using UAV uses about 430 MHz RF (radio frequency) modem and controls UAV remotely connecting through ground control system.However, as mentioned earlier, existing method of using RF modem has limitations in long distance communication.UAV control system based in Android has been made after improvements.The Android based UAV communication modules are structured to be able to connect to ground control system without control server (Fig. 1).UAV Telemetry module app is composed of communication parts that connects UAV system and the app, and GCS server that connects the app and ground control system.Communication part takes communication between smart device and Bluetooth by installing HC-06 Bluetooth module on UAV system (APM board).GCS server part takes a role of server for communication between UAV and ground control system using IP address assigned from LTE communication network base station which smart devices use.First, connection between UAV system and smart device happens through Bluetooth communication.After the connection, connection between UAV system and ground control system happens through Wi-Fi or 4G (LTE) communication, which allows data transfer between UAV system and ground control system and makes UAV remote control possible.

Telemetry module based on Android for UAV
Automatic shooting system is an image capturing device for the drones in the area's that needs image capturing and software for loading a smart camera and managing it.This system is composed of automatic shooting using the sensor of smart camera and shooting catalog management which manages filmed images and information (Fig. 2).Automatic shooting module captures images automatically by using area examination and automatic shooting stand by time and shooting catalogue management module provides function that allows you to check, delete and view the information of the images filmed by automatic shooting module (date of the filming, smart camera sensor information etc.) Figure 2. Diagram of the automatic shooting module

OpenDroneMap
This past Friday at FOSS4G in Portland, I announced the release of OpenDroneMap, a software toochain for civilian UAS/UAV image processing.The software is currently a simple fork of https://github.com/qwesda/BundelerTolls.This project like Google Streetview, Mapillary, PhotoSynth, and most small UAS(drone) postprocessing software, such as offered by senseFly.
OpenDroneMap is and open source project to unlock and make easy-to-use related computer vision techniques, so that whether from street level photos of from civilian drone imagety, the average user will be able to generate point clouds, #D surfaces models, DEMs, and orthophtography from processing unreferenced photos.

Telemetry module based on Android for UAV
The Automatic UAV Telemetry module system based on Android is developed so the communication between ground control system and drone is possible when smart camera is installed on the drone.It is developed so that it can be operated on smart devices with at least Android operating system version 4.0 developed using eclipse IDE using JAVA (Fig. 3).

CONCLUSION
Existing method of using RF modem has limited communication range of 1-2km radius, and has 10mW limitation set by the Communication Act, which gives restrictions on long distance communications.To overcome these restrictions, this research has developed UAV control system based on Android that substitutes RF modem using smart camera that can connect to network anywhere, any time with high resolution camera.Android based UAV control system has advantage of using built in high resolution camera and various sensors, 4G, blue tooth network environment to provide real-time information of UAV and camera without the Arduino communication board or control server.The UAV control system based on Android this research suggests connects UAV system and ground control system with just one smart camera and also receives 3Dlocation information and 3D position information obtained from UAV system, GPS, a gyroscope, an accelerometer, magnetic measuring sensor and other sensors, which allows the usage of real-time correction of UAV location through non-datum AT.From now on, acquisition of high resolution and high precision still images and DEM productions for evaluation part for accuracy of mapping, location of the UAV and position information and application methods for other non-datum aero triangulation methods need continuous studiesselected comparably less windy day for the aerial photography using UAV,

Figure 1 .
Figure 1.Diagram of Telemetry module

Figure 5 .
Figure 5. Test area 4.2 UAV Photogrammetric using Smart Camera This study used rotary wing UAV from fixed wing and rotary wing type.Rotary wing UAV was composed of 3DR Company's X8+ which has 8 propellers, and Figure is the UAV this research has used.Currently, there are many different types of cameras used in UAV and data of cameras and equipments are various.The camera this research has equipped is a smart camera with communication module communication function (Wi-Fi, LTE) based on Android, which was Galaxy NX of Samsung.
Figure 6.Mission Planner S/W log Brower.

From
the photographed 68 photos, ortho imagery like in the Figure and position of the filmed image by UAV was produced using OpenDroneMap(Figure 7,8).