A COMPUTER VISION APPROACH FOR DETECTION OF ASTEROIDS/COMETS IN SPACE SATELLITE IMAGES

There are many satellites orbiting around the earth capturing huge amounts of images from astronomical objects and sending them to ground stations to be stored and analyzed. This results in an increasing demand for processing and analyzing images with autonomous algorithms. NEOSSat is one of the Canadian satellites that investigates the outer space to discover new comets/asteroids in our solar system. In this paper, we proposed a method based on computer vision techniques to detect the moving objects in NEOSSat images autonomously and also estimate their path. Our method is able to detect the comet/asteroids that are only %2 different in brightness with respect to the background. Moreover, it is not limited to the linear trajectory of the object and can detect objects following a curved path. This method does not depend on the length of the trajectory as well, detecting trajectories as short as 19 pixels. Our method is computationally efficient and can be run on a laptop. We also designed a graphical user interface for our software, encouraging public usage. The proposed software won the first place of the Canadian Space Agency’s Space Apps Challenge 2019 nationwide.


INTRODUCTION
The Earth had an experience of a huge crash with an asteroid which resulted in the disappearance of the dinosaurs. While that accident happened a long time ago, a recent similar incident for Jupiter was a reminder alarm that such catastrophes could happen again to the Earth. 25 years ago, there was a collision between a comet and Jupiter which significantly affected its atmosphere for several days. This comet, Shoemaker-Levy 9 (SL9), had been discovered about one year before the accident happened (Hille, 2019). Thus, not to neglect the likely devastating impact of such events, it is vital to observe the space consistently and look for all moving objects, especially those moving near the Earth.
There are different natural and artificial objects moving in the space, including asteroids, comets, satellites, and space-based telescopes. Asteroids are minor planets, mostly composed of rocks and/or metal, and exist in the inner solar system. There are a large number of asteroids in near-earth orbits which are called Near-Earth Asteroids (NEA). Comets are space objects that mostly consist of ice and dust. When comets pass close to the Sun, they warm and release gasses as a result. This often gives a comet a fuzzy appearance with a tail.
It is very common among the general public that when they hear about space objects, they think of terminologies such as hazards and extinction. However, it is important to keep in mind that there are far more aspects regarding the investigation and study of space objects. One of the primary significance of finding asteroids and comets is to provide this opportunity for scientists to analyze these objects chemically. As most of the asteroids in the solar system are leftovers of the formation of the planets some 4.6 billion years ago, this exploration gives this opportunity to find out the probable chemical mixture of other planets (Nelson, n.d.). * Equal contribution, corresponding author Recently, scientists were amazed by observing two interstellar objects passing through the solar system. One of them was an asteroid named Oumuamua seen in 2017 (Dunford, 2019), and one of them was a comet named 2I/Borisov seen in 2019 (Garner, 2019). These are the two first unique interstellar objects detected in the solar system. Being able to find these objects, it would be possible to explore not only the solar system and other planets but also the universes outside the solar system.
To detect asteroids and comets, as moving targets, several algorithms are applied. These algorithms can be categorized as three main approaches as the basic technique, shift and add technique, and streak detection technique (Anthony, Emami, 2018).
In basic techniques, two images taken from one location in two different times are subtracted. To find out the moving targets, the method looks for peaks in the difference image. As there are a lot of noises in the sky images, recognizing moving targets becomes impossible sometimes in cases that the asteroids or comets in the images are not large or bright enough to be differentiated from noises.
The shift and add method is also referred to as the "shift and stack" and "stack and track" in the literature. This method was first introduced by (Tyson et al., 1992). The algorithm shifts the temporary images, taken from one location in the sky, relative to each other, successively, and add the shifted images together. If the shifting direction is consistent with the object's movement, the final image will show a bright spot. To check all possible movement directions of the non-stationary target, the shifting and adding process must be done in all directions. As a result, this process is computationally intensive and time consuming (Scott, Thorsteinson, 2018). Moreover, this technique is highly dependant on the brightness, size, speed, and shape of the trajectory (whether a linear or non-linear track) of the moving target in the images (Parker, Kavelaars, 2010, Anthony, Emami, 2018. The streak detection method is another technique to find moving targets like asteroids and comets in sky images. In this method, all temporary images, taken from one location in the sky, are added up together in a way that in each adding process, just the peak brightness spots in every two images have remained in the resulting image. This way, a streak which is the pattern formed by the moving target's trajectory in several images is generated (Shao et al., 2014). The most common difficulty with this method is that it is generally restricted to linear or mildly curved movement of the target object across the field of view (Gural et al., 2018). Our results show that this technique detects not only small and dim moving targets, but also detects non-linear trajectories.

NEOSSAT
As a result of recent successes in the space programs, Canada has been inspired to contribute to the space surveillance network. Recognizing that Canadian weather would reduce the effectiveness of ground-based optical telescopes, a space-based alternative was suggested and pursued (Scott, Thorsteinson, 2018). Launched in 25 th of February 2013, the Canadian NEOSSat (Near-Earth Object Surveillance Satellite) microsatellite orbits in a sun-synchronous orbit at a height of approximately 800 km above the Earth with a 100 min period (15 orbits per day). NEOSSat is able to perform space-based imaging, hence, it is well-suited to find and study potential near-earth asteroids and comets. In addition, with the use of NEOSSAt data, it is possible to search and monitor satellites and space debris. The most important feature of this space telescope is that its data is not affected by day/night and weather conditions, besides, it has a shutter enabling dark images that are used in removing instrumental errors.
NEOSSAT imagery is in the optical spectrum with an aperture of 15 cm and the field of view of 0.8 • × 0.8 • . NEOSSAT can track objects around the earth anywhere outside of a 90 • wide conical region centered on the Sun, the solar exclusion region (the red region in figure 1) (Scott, Thorsteinson, 2018). with Continuous Viewing Zone (CVZ) shown, inspired from (Scott, Thorsteinson, 2018) NEOSSat's raw data is downloaded, processed, and archived at the Canadian Space Agency (CSA) in FITS format (Canadian Space Agency, 2019). The raw astronomy images are copied automatically to CSA's Open Data archive, publicly accessible from: • FTP: Direct link to NEOSSat astronomy images on CSA FTP site.
• Government of Canada's Open Government portal: Access point from Open Government webpage.

METHODOLOGY
The main objective of this research is to find moving targets such as asteroids and comets in NEOSSAT images by the use of computer vision algorithms. The framework in this paper consists of two main steps as the data preprocessing and moving target detection. Figure 2 represents the flowchart of the proposed method. Data preprocessing: Each track is a set of temporal images that are taken while the sensor is fixed in one location. Therefore, in each track, stars are stationary on the images. Our algorithm selects each track and detect the moving target in those sets of images. Figure 3 shows the clustering results to detect various tracks in a set of data.
After selecting one track, the cleaning process is carried out through two steps. The first step is responsible for reducing systematic errors. For the cleaning and dark reduction, we used the image cleaning technique developed by (Abbasi et al., 2019). Then, to reduce noises, the bilateral noise reduction method is applied to images.
Moving target detection: After data preprocessing, the images are added up together in a way that the merged image has the maximum pixel values among all of the images in one track. Therefore, the merged image includes all the stars, noises in all images and the moving targets. Figure 4 shows an RGB merged image. While there are several streaks in this merged image, only one of them is the trajectory generated by the moving target. Other lines are noises existing in single images.
To detect lines in the merged image, we used the line segment detector (LSD) (Burns et al., 1986) algorithm. This algorithm avoids any fine-tuning as it does not require any input parameter, which is a bonus since no manual parameter change is needed to be applied in different images. Moreover, LSD has the ability to find even small line segments and this allows us to detect even low speed moving targets creating small streaks in each track. However, as shown in figure 5, several line segments are detected, most of which are either noises (shown by N in figure 5) in single images or edge lines of large stars (shown by S in figure 5).
One main issue in processing images of NEOSSAT is the existence of high energy particle strikes impacting the detector during imaging operations. These sporadic particle strikes exist in single images which makes classical image processing and detection techniques ineffective (Scott, Thorsteinson, 2018).
Here, to tackle this issue, LSD is applied to single images to detect extra lines and the resulted line segment masks are merged all in one mask. Then, to remove the most possible false positives, a dilation morphological operation is applied to the merged line mask ( figure 6). Finally, applying this mask on the image provided in figure 5 is the key to automatically detect and track the path of the asteroid/comet. Figure 6. Merged line mask undergone the dilation morphological operation, note that the asteroid path is not included in this mask.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2020, 2020 XXIV ISPRS Congress (2020 edition)

RESULTS AND DISCUSSION
In table 1 mean intensities of the merged images, mean intensities of object paths, mean ratios, the object path lengths, and image sizes of days 128 th , 292 th , and 306 th are given. The values are extracted from FITS images after they are cleaned by the approach explained in (Abbasi et al., 2019). Note that the cleaned FITS files are unsigned 16-bit integer (uint16) values which means they have 2 bytes per element, that is their range of values is between 0 to 65535.
The results show the proposed method is reliable to find moving targets in some challenging cases, as follows: • Detecting moving targets is not restricted to the trajectory shape of them across the field of view. While the path of the Near-earth Asteroid CK19D010 is not a linear streak, our algorithm is able to detect its path by creating and connecting sequential line segments (figure 7). • The proposed technique is robust in cases of having fuzzy points in images that are mostly generated by comets and are challenging to be distinguished from noises. Besides, the algorithm is capable of detecting relatively slow moving targets which are making short lines in a track of data. These features are possible in our algorithm due to the capability of the LSD algorithm to detect short and dim line segments and the use of morphological operation on the merged line mask which assures the removal of the false positives. Figure 8 shows our results in finding the first seen interstellar comet, Borisov. The approximate lengths of the paths are 19 and 26 pixels for figure 8 (a) and (b), respectively, in images with a size of 1030 × 1072 pixels (see table 1 for the information of the images).
• In most streak detection methods, human recognition of the desired streaks among the streak candidates by reviewing images is a necessity (Waszczak et al., n.d.). To reduce or even remove this necessity, it is required that the Not showing any false positives in the results above and in sets of images that do not have any moving objects on them (see figure 9), is a big step toward detecting moving targets in space images, automatically, with less needs of the observance of a person.
• The proposed software is able to detect dim moving objects. As an illustration of the capability of the code in finding dim objects, mean intensities of the merged im- The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B3-2020, 2020XXIV ISPRS Congress (2020 year-day number   (see table 1). Then the ratios of these two mean values are calculated. The ratio values for all cases are close to 1. This represents how close the intensities of the objects are to the backgrounds. For example, the results for the day 292 th of 2019 show that the object path can even be detected in cases that the mean intensity of the path has a difference of 2% with the mean intensity of the merged image. Furthermore, the proposed software is able to find the moving object path in an image with a high level of background noise. The results of day 128 th of 2019 show that while the mean intensity of the merged image is 7% more than the mean intensity of the moving object path, the object path was detected by our approach. This happened because of having a lot of bright stars and noises in the images.
These cases are resolved in this approach due to several considerations that are taken in the code. To name a few, merging images in a track helps improve the object path intensity, specially in cases of fuzzy comet appearances in the images. In addition, the LSD algorithm used in the software, is an algorithm that is sensitive to small and faint lines. Besides, the use of LSD on single images as well as the merged image in a track, enabled the proposed approach to get read of the background noises, whether their levels are lower than (like image number 2019-306), similar to (image number 2019-292), or greater than (image number 2019-128) the mean intensity of the object path (see table 1).
• Moreover, the NEOSSat data are accessible for public and our application is also an open source application which is available on our github account ( https: //github.com/CSA-Space-apps-challenge-2019/ UI_comet_NEA_finder) with all details. The developed software has a user friendly interface (figure 10), designed for users to choose their data and by one click see if there are any asteroids, comets, or moving objects in their desired sets of data. While this software is not developed specifically for discovering new objects, its purpose is to detect any moving target in the astronomical images. Thus, it is anticipated that by using the software on the various datasets, based on the occurrence chance and the appearance of undiscovered objects on images, discoveries are happening in the future. However, it is out of the scope of this study to determine if the detected objects are known or unknown objects in the sky.

CONCLUSION
In this paper, we have implemented an algorithm for detecting astronomical moving objects such as comets or asteroids on NEOSSat images which are provided by the Canadian Space Agency at their OpenData portal. The method starts with preprocessing the raw data before starting the moving target detection. Then, combining some image processing and computer vision algorithms such as spatial domain algorithms, line segment detection (LSD) and morphological operations, our method is able to find the trajectories of moving targets in space images. As the last step, based on the assumption of the object moving across the images, the probable false candidates are eliminated, leaving the true positives as the final result. The algorithm presented in this paper is robust to the brightness, detecting low brightness objects as dim as %2 of the background brightness. It can detect the short line streaks as small as 19 pixels in length, and even non-linear trajectories which are estimated by linear segments. The code and its user-friendly graphical interface are open source and available to the public, which can be run on a personal computer. The proposed software won the Canadian grand prize of the Canadian Space Agency's Space Apps Challenge 2019.