CONSTRUCTION AND TEST OF BIO-INSPIRED IMAGING POLARIZATION NAVIGATION PROTOTYPE

Bio-inspired polarization navigation is a promising navigation method inspired by insects’ autonomous foraging and homing behaviour. Many insects acquire their spatial orientation by sensing the polarization pattern of the skylight. We propose utilization of solar meridian in the polarized skylight as an orientation cue because of its significant features. Using its features, we then design and construct an imaging polarization navigation prototype. The prototype consists of a field-division polarization imaging sensor, the corresponding software, an interface, and the solar-meridian recognizing and measurement algorithm. The field-division polarization imaging sensor is the core component of the prototype and acquires polarized intensity images. To adapt to the demand of real-time on navigation system, we then propose an optimized real-time polarization image processing and pattern recognition algorithm based on Hough transform. The azimuth measurement accuracy of the sensor is then calibrated using a facility that is able to get higher azimuth accuracy by measurement of the star light. To verify the navigation capability of the developed system, we use a dynamic experiment, where the prototype is installed on the top of a vehicle and its navigation performance is compared with GNSS. * Corresponding author


INTRODUCTION
Many insects are able to navigate based on sensing orientation cues in the skylight, see e.g., [Heinze, 2020]. Polarization navigation is suitable for entities located in a strange environment with no or limited landmarks and/or other navigation aids. Instances of such environments include a locusts navigating its way in the desert, and Mars rover [Chahl, 2011] [Thakoor, 2004].
Polarized skylight is one of the primary and ideal navigation cues [Pegel, 2018]. In particular, it is demonstrated that polarized skylight is stable under the clear, cloudy and overcast sky [Hegedüs, 2007] [Pomozi, 2001] [Suhai, 2014]. Several prototypes which use photodiodes as photoelectric converters have been developed based on the polarization-sensitive behaviour of insects [Chu, 2007]. To enhance the navigation precision and its anti-interference behaviour, several imaging polarization navigation methods are proposed in the literature. In [Ma, 2015], the symmetry algorithm is used to measure the orientation of the solar meridian. Hough Transform (HT) is employed in [Lu, 2015] to capture and recognize the features of the solar meridian in the Angle of Polarization (AoP) image. The above algorithm however is computationally complex as it includes several non-linear operations to obtain the AoP image. Efficient polarization acquiring sensors and real time orientation determination algorithm are necessary for navigation systems.
Several calibration methods exist however they usually calibrate the precision of the polarization measurement. Instead of the accuracy calibration of the orientation measurement, [Chu, 2008] [Powell, 2013] investigate precision calibration of AoP. In these calibration methods, ideal polarized source is employed to calibrate the point-source polarization navigation sensors, which use photo-diodes as photoelectric converter. This method calibrates the measurement precision of the sensor instead of precision and accuracy of navigation because real light source, polarized skylight, is not ideal. On the other hand, point light source cannot simulate polarized skylight pattern for imaging sensors. It is significant to calibrate the measurement accuracy of the imaging polarization prototype under the real sky dome.
In [Chu, 2008] [Lambrinos, 2000] [Fan, 2011], several bio-inspired polarization navigation prototypes which use photo-diodes as optic-electric converters are investigated for moving entities to evaluate their navigation ability. The imaging polarization navigation algorithms are test in [Stürzl, 2012] [Wang 2014] but the images are processed offline on computers. It is easy to reach adequate refresh rate for navigation because photo-diodes output less data. On the contrary, the imaging polarization navigation sensors generate large amount of data to process. Real time is a critical issue for navigation. It is important to test a real-time navigation sensor on a moving carrier.
In this paper, we develop an imaging polarization navigation prototype including a field-division polarization imaging sensor, the accompanied software, the interface, and the solar-meridian recognizing and measurement algorithm. An optimized algorithm for DSP which can measure rapidly is proposed. To calibrate the orientation accuracy of the sensor, a calibration system is also developed that utilizes the real skylight. The absolute azimuth is measured from the star light. We also conduct a dynamic experiment in which the prototype is installed on the top of a running car and the measurements are compared with the measurement of GNSS receiver to verify its navigation capability.

SOLAR MERIDIAN-ORIENTATION CUE IN THE SKYLIGHT
Mie and Rayleigh scattering are major types of scattering in the atmosphere. Mie scattering is caused by particles, whose scale is similar with wavelength of light, whereas Rayleigh scattering is caused by particles whose scale is far smaller than wavelength of light such as gas molecules. The polarization phenomenon in skylight is caused by Rayleigh scattering. Because gas molecules are widespread, Rayleigh scattering is stable and distributes regularly in the atmosphere [Lu, 2015]. The stability of the pattern in polarized skylight under different weather conditions, such as clear, cloudy and overcast sky was demonstrated in [Zhao, 2018]. The single-scattering Rayleigh (SSR) model can describe a large portion of the sky and these areas can be utilized for polarization navigation as Figure.1 [Suhai, 2014]. Thus, we use the SSR model to simplify the distribution pattern of polarized skylight. The scattered light became polarized after sunlight is scattered by particle at point (h p , φ p ) in the celestial sphere. The sunlight is from (h s , φ s ). From Rayleigh theory, polarized part in sunlight that is parallel to the surface formed by sunlight s and scattered light h would be partly extinct. Accordingly, the polarization direction of the scattered light h become orthogonal to the surface formed by sunlight s and scattered light h. The AoP of skylight is defined as the angle ɑ clockwise from the local meridian that is formed by zenith and scattered light. From spherical trigonometry, the angle ɑ can be expressed as where h p , h s =the elevation of scattered light and the sun, respectively φp, φ s =the angular distance between the x-axis and the projection of the scattered beam and the projection of the sun, respectively.
Through analysis, we can conclude three significant features that solar meridian: E-vector of 90°, straight line, and through the principal point form Equation (1).

OPTICAL, MECHANICAL, AND ELECTRONIC DESIGN
The imaging polarization navigation sensor can acquire polarized image of skylight, process the images, recognize and measure the solar meridian in the sky successively. Azimuth of the solar meridian that is the output of the prototype is send to PC client throughout Ethernet link. Imaging polarization navigation prototype consists of aluminium shell, image sensors, filters module, and lenses as Figure 2. The filter module includes blue filers and polarizers with the dowel chords. Since there is a stronger Rayleigh scattering the shorter wave length is, filters whose wavelength from 400 nm to 550 nm are chosen. Usage of polarizers in 0, 45°, and 90° is in consideration of deduce floating-point multiplication operations than usage of polarizers in 0, 60°, and 120°. According to the analysis in , the polarizers arrangement in 0, 60°, and 120° can achieve optimal condition number. Meanwhile, the condition number of the arrangement in 0, 45°, and 90° has no obvious ascend. Despite the three imaging system are same, differences must be involved when assembling. Zhang's calibration is utilized to calibrate and to correct the differences of their intrinsic and extrinsic parameters between three imaging systems [Zhang, 2000]. To make them consistency, we conduct two stereo calibrations between A and B imaging systems and between A and C imaging systems. Then, B and C systems are corrected to A system that is reference coordinates of the sensor. The consistency of response of COMSs is calibrated and corrected using a uniform source of light. [Lu, 2016] The prototype is equipped the VCSBC Quadro system by Vision Component as Figure 3. It has four 1/3", 640×752 CMOS sensors. FPGA and DSP are integrated on broad. FPGA controls sensors to acquire images which then buffered into RAMs. DSP processes images, recognizes patters, and solves the heading of the sensor. To fasten the rate of the procedure, the algorithm was adapted in the following for fixed-point DSP which is more common at present. When it works, the sensor and PC make up a TCP/IP network throughout an Ethernet cable. The control command, images, and azimuth calculated by the sensor are all transfer throughout this data link. In the network, the sensor working as a server always waits for commands send from PC when it is idle. It executes command and returns the output when receiving commands such as calculating azimuth.

REAL-TIME IMAGING AND PATTERN RECOGNITION ALGORITHM
The core function of the algorithm runs in DSP is process images data to azimuth. The polarized intensity images are acquired and processed. In tradition method, these images are transfer to AoP image in which the pattern is recognized and azimuth is measured [Ma, 2015] [Lu, 2015]. It involves a large amount of computation because the several nonlinear operations as Equation (3). In this work, pattern recognition and azimuth measurement is in I, Q, and U image to descend the computation.  According to the features of solar meridian above, we designed, implemented, and optimized the polarization imaging and solar meridian recognition algorithm as Figure 4. The algorithm can reduce the operation times of the processor to accelerate the solve rate. The polarization light image can be expressed in the Stokes parameters which can be solve as Equation (2).
where I 0 , I 45 , I 90 = intensity images acquired by cameras with 0, 45°, and 90° polarizers, respectively. I in , Q in , U in = stokes parameters images calculated from intensity images β = angle from x-axis of the sensor From Figure 1 and Equation 1, the solar meridian is formed by the skylight and the z-axis. Features of solar meridian are expressed in the AoP (Angle of Polarization) image. Calculation of AoP image as Equation 3 is the traditional method. However, there are a large number of non-linear operations such as division and arctangent. To accelerate the solve rate, we can take advantage of the Stokes parameters, which form a linear vector in a given linear transformation (4). The local meridian coordinates are obtained by rotating the measurement coordinates by β. Since cos2β and sin2β are only determined by the location of the pixel (i, j), their values of a certain pixel can be obtained by look up the table that is stored in the memory. Therefore, the Stokes parameters about the local meridian can be expressed as: According to the features of the polarized skylight, the AoPs on the solar meridian are around 90°. The binary image of solar meridian can be gathered by Equation 5. To fasten the process rate, Equation (5) can be represented by Equation (6) which contains only several simple operations. Since polarization is expressed in the Stokes parameters, the operation of (5) is equivalence to Equation (6).
To search the solar meridian, we should find a straight line through principle point according to its third feature in binary images generated by Equation (6). A modified Hough transform searches the solar meridian in the [Q', U'] space instead of the tradition method that searches it in the AoP images. Commonly, Hough transform transfer 2D points in image space to 2D parametric space. In this work, the parametric space can reduce to 1D parametric space as Equation (7), (8) .
where X k are the coordinates of the binary point (i k ', j k ') with reference to the principal point.
After the solar meridian is recognized, the orientation of the solar meridian is calculated through the Equation (9) in parametric space [LU, 2017]. Though we can get the angle of lines by Hough transform directly, we use the weighted average of points of Q and U values on the solar meridian. The Hough transform would be inefficient if its resolution is set too high. Thus, the resolution of the Hough transform is set to 1° to ensure it can cluster an effective peak. Meanwhile, Equation (9) is to ensure the measurement precision of the azimuth of the solar meridian.

ACCURACY CALIBRATION
The calibration of the sensor contains of calibration for the consistency between the three optical systems and for orientation measurement. The calibration for the consistency is discussed above. In this section, we propose an accuracy calibration using real sky dome as light source for the imaging polarization navigation sensor. At present, ideal polarized source is employed to calibrate the point-source polarization navigation sensors, which use photo-diodes as photoelectric converter. This method calibrates the measurement precision and accuracy of the sensor instead of navigation precision and accuracy because the light source, polarized skylight, is not ideal. The other reason to propose calibration method using real sky dome as light source is that it is difficult to generate the polarized skylight pattern in laboratory using point polarization source.
The key condition of to calibrate the imaging polarization navigation sensor under the skylight is to measure and calculate sun vector in sensor coordinate. Two problems should be solved. First, How to get the topocentric coordinate in high accuracy? Second, How to transfer topocentric to sensor coordinate? The horizontal base surface of topocentric coordinate can be acquired by measuring plumb line conveniently. The azimuth surface is hard to get. Magnetic azimuth can be measured easily but is with a low accuracy about 1°. The accuracy of GNSS azimuth relies on the length of the measurement baseline meanwhile the large, fine-straightness baseline is hard to manufacture. In the article, we utilize star light as orientation reference. We use theodolite to aim at the star, to measure the azimuth, and to transfer them to sensor coordinate. The calibration facility is as Figure 6. The azimuth measurement principle of the facility is as Figure 7. The theodolite can get absolute azimuth reference by aiming at sight line of the Polaris. Then the absolute azimuth of the sensor can be measure by theodolite aiming at the prism fixed on the sensor because unidirectional sensitive character of the prism. The prism is 180° prism as Figure 8. From its optical path, it is sensitive in the horizontal direction but insensitive in the vertical direction. In spite of height difference between the theodolite and the sensor, the method can measure the azimuth instead of the elevation.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition) This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B1-2020-323-2020 | © Authors 2020. CC BY 4.0 License. orientation parameters. Positions of the sun and the earth can be calculated from JPL ephemeris according to the measurement time. In this work, we utilize DE421 ephemeris to calculate the earth and the sun position as Equation (9) To get the azimuth and elevation of bodies in the topocentric reference, the transform matrix from ECI (Earth Centered Inertial coordinate system) to ECEF (Earth Centered Earth Fixed coordinate system) can be calculated according to IERS (International Earth Rotation and Reference Systems service) parameter and the measurement time as Equation (11) (12). The calculation procedure is as Figure 9.   Figure 9 Calculation procedure of azimuth, elevation of the sun and the Polaris

DYNAMIC TEST
Dynamic test was conducted to verify the performance in WeiLaiKeXueCheng, Beijing, around 40.133°N, 116.471°E. The facility arrangement is shown as Figure 13. We fixed the prototype and GNSS receiver on the top of a moving vehicle in a particular route as Figure 10 around 30 km/h. The route is a closed quadrilateral and three sides are straight lines (north, east, and south). The west is an arc. The heading can be gathered by the prototype and GNSS. We can utilize the movement of the vehicle to measure its heading using the difference of the position sequence as Equation (16) where lon(t)=longitude at t from GNSS lat(t)=latitude at t from GNSS Figure 10. Route of the vehicle in the experiment The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition) This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B1-2020-323-2020 | © Authors 2020. CC BY 4.0 License. Figure 11. Comparison between GPS heading and results of the sensor Refresh rate of GNSS is about 1 Hz. Figure 11 shows the comparison between the two methods. The sensor and GNSS output a three steps like curve when vehicle moves on the north, east, and south sides. They output a little and a bigger curve when the vehicle moves on southeast corner and west side, respectively. The heading plots in figure can well reflect the movement of the vehicle. It is a coincidence between results of the polarization navigation sensor as the green line and results of GNSS as the blue line. The sensor has ability to measure in movement. Through the test, the prototype can achieve the rate about 20Hz which is far faster than GNSS and can satisfy most application scenarios of navigation.

CONCLUSION
In this paper an imaging polarization navigation prototype was constructed and tested through a dynamic experiment. The prototype acquires, process, and rapidly calculates polarized skylight data using an optimized imaging and pattern recognition algorithm. The accuracy and precision of the sensor was calibrated under the real skylight in which higher accuracy orientation reference was measured using celestial navigation method. A dynamic test was also conducted in which the prototype was mounted on the top of a moving vehicle to compare with the measurement of GNSS receiver and verify its navigation capability.
Our results show that the designed system is able to support real time navigation. Its autonomous measurement and the fact that it does not need direct sunlight, the proposed polarization navigation can provide navigation aid under extreme condition such as on the surface of terrestrial bodies when the sunlight is blocked. Rovers working in such environment always depend on sun sensors to measure the azimuth cue, this may result in inaccurate navigation while the sun is shielded. Since it is unnecessary that the sun directly irradiates the sensor, the polarization navigation can keep working under extreme conditions, for instance, where the sun is shielded by terrains, or methane clouds on Mars, or even before the dawn. The proposed method has potential applications in the field of terrestrial body exploration with atmosphere such as Venus, Mars, Titan.