UNDERWATER IMAGE ENHANCEMENT BASED ON POLARIZATION IMAGING

The need of high-quality underwater imaging is obviously required in many underwater applications. For example, underwater archaeology, underwater ecological research, underwater object detection and tracking. This paper presents a joint enhancing and denoising scheme for an image taken in underwater conditions. Conventional image enhancing methods may amplify the noise depending on the distance and density of the particles in the water. To suppress the noise and improve the enhancement performance, an imaging model is modified by adding the process of amplifying the noise in underwater conditions. This model offers depthchromaticity compensation regularization for the transmission map and chromaticity-depth compensation regularization for enhancing the image. The proposed iterative underwater image enhancing method with polarization uses these two joint regularization schemes and the relationship between the transmission map and enhanced irradiance image. The transmission map and irradiance image are used to promote each other. To verify the effectiveness of the algorithm, polarizing images of different scenes in different conditions are collected. Different algorithms are applied to the original images. Experimental results demonstrate that the proposed scheme increases visibility in extreme conditions without amplifying the noise. * Corresponding author


INTRODUCTION
Underwater imaging technology has been widely and deeply applied in many fields, such as seabed topography exploration, marine biological monitoring, underwater resource exploration and underwater archaeology. The transmission of light in water is a physical process of interaction between light and components in water medium, and the imaging effect of optical imaging system is directly related to the medium environment of water. Under the influence of attenuation and scattering in the process of underwater light propagation, the underwater visibility is generally 20 m in pure water area and only 5 m in turbid water. In the process of underwater imaging, water has obvious selectivity for light absorption. The longer the wavelength is, the larger the light absorption attenuation coefficient is, and the shorter the visible distance is. Therefore, underwater images usually show a short wavelength blue-green tone, with obvious colour distortion. The scattering attenuation of water can be divided into forward scattering and backward scattering. Forward scattering results in blurred image details and missing texture information. The backscatter results in the decrease of image contrast and the white effect on the image surface and the decrease of signal-tonoise ratio. In view of the problems of underwater image processing, researchers proposed a variety of underwater image processing algorithms.

RELATED WORK
At present, the processing methods for enhancing or restoring underwater images can be roughly divided into two types (Wang et al., 2019): non-physical model based image enhancement methods and physical model based image restoration methods. Image enhancement technology does not need to consider the process and model of image imaging too much. It can be called non-physical model method. This method tries to improve the quality of underwater image through simple image processing, and improve the visual quality by adjusting the pixel value of image, but the implementation process is more complex. This kind of method improves the image quality by directly adjusting the image pixel value, and does not consider the physical process of underwater image degradation. In the early research of underwater image enhancement technology, some traditional image enhancement algorithms in the air are often directly applied to underwater image processing. The traditional image enhancement algorithms can be divided into spatial domain method (G. Hou et al., 2018) (Ancuti et al., 2017) and frequency domain method (Jian et al., 2017) (Vasamsetti, et al., 2017). The spatial domain method is to directly process the pixel points in the image, using the method of gray mapping, such as selecting the appropriate mapping transformation to increase the contrast of the image, improve the gray level of the image and so on. The frequency domain method is an indirect image processing method. It uses the transformation technology to map the image to a certain transformation domain, then uses the unique properties of the transformation domain to carry out some filtering processing, and then reversely transforms to the space domain, that is, to get the enhanced image. The traditional spatial domain enhancement algorithms that are often applied to underwater image include histogram equalization (Zuiderveld, 1994), gray-scale world hypothesis and white balance algorithm. The frequency domain enhancement algorithms include Fourier transform, wavelet transform and filtering technology, mainly including low-pass filtering, high-pass filtering and homomorphic filtering (Grigoryan et al., 2018). The tasks of image defogging, deblurring and blind image restoration based on convolution network are similar to underwater image enhancement (Li et al., 2018) (Wang et al., 2017). But the key point lies in the acquisition of training set and the generalization ability of convolution model. For such a complex underwater environment, it is difficult to train a network with strong generalization ability. At the same time, it is difficult to find the data set. In the method of underwater image restoration based on physical model, a suitable degradation model of underwater image should be established for underwater image, and then the degradation process of underwater image can be reconstructed to restore the original underwater image without degradation in the ideal state. The typical method of recovering underwater image based on physical model is underwater image restoration based on dark channel prior and based on polarization. He proposed a dark channel prior (DCP) algorithm (He et al., 2009). In the image without fog, the colour of each local area in the image is more distinct. According to the principle of three primary colours of an image, the value of at least one colour channel in each region is very low. The image composed of the minimum value of each channel in the image is called a dark channel image. If there is fog in the image, because the fog is gray, the value of each channel is high, that is, the value of dark channel is also high. Applying DCP algorithm directly to underwater image enhancement cannot get a good enhancement effect. But it can be realized by improving the algorithm. The polarization based method (Huang et al., 2016) uses images of the same scene captured by different degrees of polarization to restore underwater images. These images are obtained by rotating a polarization lens fixed on the camera. For example, Schechner used polarization related to backscattered light to estimate transmittance (Schechner et al., 2007). It is proved effective in recovering the image visibility.

UNDERWATER IMAGING
Affected by suspended particulates in the water, a captured image consists of forward scattered light and back scattered light passing through the water. Both of them reduce the final quality of the image and decline the visibility. The observed intensity can be described as: where L(x) denotes the radiance of a clear scene seen at a spatial location x, B ∞ is the intensity of the water at infinity. I(x) denotes the degraded version of L(x) by scattering. Transmission map t(x) represents the rate of transmission through the medium describing the scattering and absorbing (forward scattering) of particles in the atmosphere. The transmission map, t(x), can be expressed as an exponential function of the distance: (2) where β denotes the optical density of the particles in the water, which is assumed to be a constant at different wavelength. The depth map z(x) indicates the distance from the camera to a scene point at a spatial location x in the image. The suspended particulates in water will scatter and absorb the light from the scene and radiation source, which is important factor for the degradation. To acquire the scene radiance, t should be acquired. Because of the polarization effect of the suspended particulates, the irradiance reflected from the scene through the water could be thought as unpolarized light. Based on the polarized scattering effects, t can be estimated as: Where I 0 and I 90 are images acquired through polarizers at 0°a nd 90°. However, as the particles density increases, conditional performance drops quickly. Visibility enhancement often leads to amplification of non-local noise. In this paper, we will focus on improving the visibility of the image as well as suspending the noise.
Most previous enhancing methods are based on imaging models in the ideal hazy conditions. However, in the actual situation, the target image is infected by the imaging noise inevitably. We assume that the noise appears when the irradiance of the scene is captured by the equipment, so the noise would only affect the final image we take. Taking the noise into consideration, we can easily reach that the model: where n denotes the imaging noise, L0 is still the initial irradiance of the scene. As we take the noise into account, we can get the result L0 after enhancing: We can define the added noise as ε: is fixed, the noise would increase exponentially as z (or β) grows.

PROPOSED METHOD
In order to wipe out the noise in the scene after enhancement, the most important problem is to solve the problem of local image denoising according to the specific particles degree and distance. So the transmission map t contains both the information of distance and the density of particles should be the guidance. The use of the guidance of depth map improves the bilateral filter for denoising.

Guided Weighted Regularization for Transmission Optimization
From the statistical characteristics, for most outdoor clear scene in the local region of image, the depth information is approximately consistent; and when the transmittance difference of two pixels is large, reflecting the larger difference in depth, the corresponding colour between two pixel jumps becomes more obvious. Based on the statistic characteristics, we will try to independently deal with the scenes of different depths. We present a feedback optimization to guide the weighted regularization method to optimize the transmission coefficients. In order to guide the weighted regularization, the regular expression is as follows: (7) and (8), W is the guided weighted function, i is for the color channel, (x,y) is for the two-dimensional pixel point, is the standard deviation. In the local region of the image, due to the influence of noise, if the two pixels transmission difference is large, the colour of the two pixels of the corresponding enhancing image differ obviously. At the same time, the two pixels in the image edge region jump sharply, equally. Therefore, this paper proposes that the directed weighted regularization term can effectively suppress the noise amplification, and preserve the edge details of the image. So the total objective function is： (9) According to (3), the initial estimated value is t0, w(.) is the local window of the center pixel, λ is the parameter to be adjusted. The optimization process is satisfied: arg min ( ) opt t t E t  (10) This optimization process is not so complicated, and we can get the optimized transmission map by optimizing method as Gradient Descent. Although the result is optical locally, it is proved to be effective in our following experiments. And until now we can get the optimized transmission map.

Adaptive Joint Bilateral filtering for Irradiance Refinement
When imaging in dense foggy weather or in long distance, the image boundary distortion (artifacts) is often caused by the serious disturbance of the atmosphere. It is known from the section above that the optimized transmission coefficient image can effectively suppress the noise and distortion. Joint bilateral filter considers the space and range difference between the neighboring pixels as the weight of the filter, and in our way to enhance images, we find the difference of neighboring pixels in the transmission map should, after the explain above, also be considered in the expression of the weight of filter. In order to remove of the scattering and retain the edge of the final result, we use the optimized transmission coefficient as the feedback to perform the adaptive joint bilateral filter to improve the visual effect. A large б blurs more, that is, it combines values from more distant image locations. M is for the normalization. f is the adaptive transmission function. To the close of the scene, the distortion degree is small, so f should be large. As a result, the close region will get a little blur to reserve the edge; to the scenes where distortion is more serious, f should be small and clear up the noise. So as we know it is a negative correlation between 0 and 1. So we can set f as: The square calculation could amplify the difference between the near and far regions. Different from the traditional bilateral filtering only by the radiation intensity of single source image and spatial domain information, our adaptive joint bilateral using optimization of transmission coefficient of the image as a guide, at the same time, the introduction of adaptive transmission function could be more precise to suppress noise amplification and edge distortion at the same time, effectively keep the scene of the edge and texture information as well.

Adaptive colour correction
Restoring images degraded by the underwater environment is a challenge. Part of the reason is that underwater light transmission is still affected by absorption degradation. The main consequence of this degradation process is the color distortion of each beam wavelength. Absorption that does not exist in the atmosphere because it is considered transparent. But the water affects the way the colour disappears in the underwater scene. The red channel loses its intensity rapidly, while the green and blue channels receive more intensity. In this case, whether the image is degraded or not, there is almost always a low-intensity colour channel (red channel). This problem can be transformed into underwater environment through certain adaptability. To do this, we can rearrange the original model and write: Given an estimate B = (B r , B g , B b ) for water light, it is assumed that the transmittance is constant, and under the red channel assumption (16), t (x) can be modified as: Bring t into (1) to get the final result. However, before extending the model directly to include the red channel, the speed of different wavelengths in the light is not considered. In order to incorporate this factor into our model, we extend the model (14) to cover three transmission maps, one for each colour component. In (15), we need to estimate three components of water light and three transmission graphs, one for each colour. However, these three matrices are not independent. In fact, only one matrix and two scalar numbers need to be estimated. We have Suppose we have estimated the water light and attenuation coefficients β g / β r and β b / β r . Bring t into (1) to get the irridiance. 2. Colour correction: transmission t←transmission t'; 3: Conclude the radiance L ←transmission t'; 4: repeat 5: optimize t using guided weighted regularization. 6: until there is no change to t 7: Improve and apply the joint bilateral filter to optimize L and we get the final optimized radiance Lopt. 8. return Lopt

EXPERIMENT
In this paper, the polarization image database of underwater objects is established. For each scene, a linear polarizer is used to capture two images simultaneously at an angle of 0°and 90°. For each kind of object, more than 100 groups of polarized images are collected under different shadow conditions. This database provides enough data to support our experiment. By using the polarimetric underwater image database, the image enhancement methods proposed in this paper are compared with the latest methods: CLAHE (Zuiderveld, 1994), GC (Deng, 2011), HE (Hummel, 1977), ICM (Iqbal et al., 2007), RD (Ghani et al., 2014), RGHS , and UCM (Iqbal et al., 2010). The results of each method are given respectively. Fig.4 shows scenes in water with different turbidity degrees and the underwater image results of the proposed method and other conventional methods. From the collected original images, it can be clearly noted that the objects become blurred and the image quality is greatly reduced. GC, ICM and RD barely improved contrast. CLAHE, RGHS and UCM slightly improve the contrast, but in some extremely dark areas, the contrast enhancement is insufficient. He improves the contrast clearly but it is inferior to the method in this paper. In contrast, this method has a good effect in enhancing and retaining details.
The results show that under extreme conditions, the performance of polarization method is better than that of panchromatic image method. This method effectively improves the contrast without sacrificing texture and edge information.
There is no obvious colour distortion in the image, and the noise is not significantly amplified in this process. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2020, 2020 XXIV ISPRS Congress (2020 edition) The experimental results in Figure 3 show that CLAHE and GC do not significantly improve contrast. In Figure 4, HE, ICM, RD RGHS and UCM methods increase image contrast. However, there is obvious colour distortion and noise amplification. In contrast, this method gets better results and keeps the sharpness of the object edges.
Only the subjective evaluation is not convincing, so the objective evaluation index is also needed. In the image enhancement task, contrast is the most important evaluation index. In addition to contrast, enhancement methods should also avoid artifacts. In recent years, there are many blind image quality evaluation methods. Therefore, we use two blind image quality evaluation methods in the evaluation:(a) ILNIQE (Zhang, et al., 2015) uses five types of features, including MSCN based features, MSCN based product features, gradient based features, log Gabor response based features and color based features. These features are applicable to the task of completely blind image quality evaluation. The lower the score, the better the performance. (b) NSS (Fang, et al., 2014) developed a non-reference quality measurement method based on natural scene statistics, which is specially used for contrast distortion image. NSS model is based on moment and entropy.
Then, the quality of contrast distortion image is evaluated according to the non-naturalness of the image represented by the deviation from NSS model. Higher marks mean better performance. We calculated the non-reference perceptual image evaluation ILNIQE and NSS of the results in Fig.3 and Fig.4 as shown in Fig5-8, respectively. Two evaluation results show that the proposed underwater image enhancement method performs better in most scenes, especially in the case of heavy blur. Although the evaluation results show that the method in this paper has good results, we have to admit that there are still some problems in the existing evaluation methods: 1. It is not sensitive to the impact of noise, 2. The colour distortion is not considered properly.

CONCLUSION
In this paper, an underwater imaging model considering the process of noise amplification is proposed, and the relationship between the noise level after image enhancement and scene distance and water particle concentration is found. On this basis, we propose an iterative image enhancement process to suppress the noise generated in the enhancement process, which includes two weighted regularization processes. Considering the problem of colour balance caused by the different absorption rate of different wavelengths in water, the optimization of red channel transmission map is used to improve the colour balance of underwater image. Experiments show that the method can enhance underwater image well under different scenes and particles concentrations. At the same time, our method suppresses the noise, keeps the details and restores the colour.