TOWARDS BETTER COASTAL MAPPING USING FUSION OF HIGH TEMPORAL SENTINEL-2 AND PLANETSCOPE-2 IMAGERIES: 12 BANDS AT 3 M THROUGH NEURAL NETWORK MODELLING

Coastal interfaces are subject to an unprecedented rate of risks, gathering waves and rainfalls’ hazards, human assets’ densification, sea-level rise and precipitation intensification. Their sound management requires iterative observation at the highest possible spatial resolution. Sentinel-2 (S-2), provided with 13 spectral bands, datasets leverage high temporal resolution (one week) but spatial resolution (from 60 to 10 m) often remains too coarse to finely classify and monitor the coastal patches. PlanetScope-2 (PS-2) imagery benefits from very high temporal resolution (<one week) and high spatial resolution (3 m) for its blue-green-red-nearinfrared dataset. This research paper proposes to, first, downscale 12 S-2 bands (cirrus S10 being evicted) by using neural network (NN) regressions built on the 4 PS-2 bands following two methods, and second, evaluate the NN classification performance of the 12-band datasets at 3 m for mapping 8 common coastal classes on a representative site (Brittany, France). Straightforward and stepwise downscaling procedures, respectively based on 12 and 22 NN regressions, generated very good performances (Rtest=0.92 ± 0.02 and 0.95 ± 0.01, respectively). The 3-m NN classifications were considerably improved by the number of spectral bands (overall accuracy, OA, of the 4 bands: 48.12%) but also the precision of the downscaling (OA of the straightforward and stepwise downscaling: 75.25% and 93.57%, respectively). For the best classification, examination of the contribution of the individual bands revealed that S5, S7, S1, S9, S6 and S8A were meaningful (62.42, 55.02, 50.82, 46.4, 45.1, 31.02%, respectively), contrary to S12, S11 and S12 (12.47, 0 and 0%, respectively). * Corresponding author


Downscaling Satellite Optical Remote Sensing Imagery
The downscaling procedure has allowed spectrally-riched, regionally-acquired, coarse-scaled imageries to be refined at higher spatial resolution. The MODIS chlorophyll-a level-3 product at 4 km pixel size has been downscaled to 30 m using Landsat-8 OLI for complex coastal water monitoring (Fu et al., 2018). In turn, Landsat-8 OLI 30-m multispectral bands have been pansharpened to 15 m, then downscaled to 5.8 m (using ZiYuan-3 imagery) for better extracting coastlines (Wang et al., 2018). The bathymetry mapping (without in situ data) has been enhanced from Landsat-8 to very HSR (VHSR) satellite sensors (Gaofen-1/2, ZiYuan-3, WorldView-2, Liu et al., 2021). S-2 MSI 60-m and 20-m bands have been successfully downscaled to 10 m for improving LULC mapping (Zheng et al., 2017). The S-2 coastal band, downscaled from 60 m to 10 m, was deemed very promising to both deepen the water depth and seagrass mapping in Mediterranean Sea (Poursanidis et al., 2019). Recently, Landsat-8 OLI 15-m panchromatic band has been downscaled to 3-m PS Dove imagery, then used for pansharpening the 30-m multispectral bands in order to map bathymetry (Gabr et al., 2020).

Sentinel-2 and PlanetScope-2 fusion
Landsat-8/9 and S-2 bands or by-products constitute tangible advances for coastal mapping when downscaled with higher commercial optical spaceborne sensors. Given its comparable high temporal resolution (HTR) with Landsat-8/9 and S-2, the PS constellation could be considered as a good candidate for downscaling both freely available HSR NASA and ESA imagery. Contrary to SPOT of finer imagery series, PS leverages greater global distribution and cheaper prices (even free for granted scientists, Planet Team, 2017). This research study innovatively proposes to produce 12 bands at 3 m spatial resolution for coastal mapping by fusing both HTR (two 60-m, six 20-m and four 10-m) S-2 and (four 3-m) PS-2 (PS-2) imageries through neural network (NN) modelling. First, the NN regression downscaling will be optimized by testing the statistical reliability of the straightforward versus the stepwise approaches. Second, the NN classification of a complex, thus representative, coastal area, composed of 8 common classes (Figure 1), will be quantified at the scene scale for the two downscaling techniques and also for the original PS-2 dataset, for the sake of comparison.

Study Area
The investigated coastal zone (48°37'N; 2°7'W) is located along the Emerald Coast in Brittany (France). It features coastal shallow and optically-deep seawaters, muddy estuaries, sandy beaches, rocky cliffs, resort cities, crop fields and semi-natural vegetated areas ( Figure 1 and Table 1). Subject to a megatidal regime, the intertidal zone alternates with emersion and immersion phases, entailing sediment class features to be examined in their wet and dry status. For the sake of transferability, the generic terms of crop and soil encompass all features of the herbaceous stratum, and unvegetated bare but living layer, respectively.

PlanetScope-2:
The PS-2 dataset was derived from a Dove Classic nanosatellite (1-day revisit and a frame size of 24 km × 8 km). The Dove Classic constellation was launched in 2016 and 2017. The imagery was acquired on 09 October 2019 at 10 h 49 min 48 sec (UTC), then orthorectified and radiometrically-corrected at the surface reflectance (the equivalent of the BOA). The resulting dataset consisted of 4 bands at 3-m pixel size (Table 3) Table 3. Spectral specificities of the PlanetScope-2 Dove.

Imagery Processings
Downscaling Process: The entire scene was first divided into: • a calibration, • a validation, and • a test sub-datasets, whose their number of pixels was equalized and randomized for each spectral band.
Two approaches of imagery fusion based on NN regression were designed and evaluated.
The first procedure, coined as straightforward, sought for predicting the 12 S-2 bands from the 4 PS-2 bands, regardless of the S-2 native resolution ( Figure 2).
The activation function was a function of transformation, defined as a hyperbolic tangent function (TanH), scaling values between -1 and 1 bounds: , where z = a linear combination of the predictors (X).

Pixel-wise Classification:
The 8 classes were each represented by 1000 pixels, randomly split into calibration and validation datasets. The "calval" pixels were selected based on a pansharpened multispectral Pléiades-1 imagery acquired on 22 October 2019 at 11 h 25 min 49s (UTC, James et al., 2020). The 500-pixeled calibration datasets were used to construct NN learners defined as one-neuroned two-layered perceptrons provided with a logistic (sigmoid) activation function. Following the classification procedure, the 500-pixeled validation datasets were used to quantify the overall accuracies (OAs) derived from the PS-2, straightforward-and stepwisedownscaled S-2 confusion matrices.

RESULTS AND DISCUSSION
The fusion of the 12-band S-2 and 4-band PS-2 yielded, despite a slight difference, very satisfactory results for both NN downscaling procedures. The classification of the 8-class coastal scene was much better with the 12-band than the 4-band dataset at 3 m.

Performance of the Downscaling Procedure
The straightforward downscaling necessitated 12 NN regressions, compared to 22 ones for the stepwise approach, what implied a faster process.

Straightforward Downscaling:
The lowest regression performance, even satisfactory, was tied with the coastal band (B1, R 2 test=0.79), followed by B5 (R 2 test=0.83), and B2 (R 2 test=0.84). The regressions of the remaining bands bottomed at 0.91 (B9) and topped at 0.99 (B8). The overall mean and standard deviation reached 0.92 ± 0.02. The 60-m, 20-m and 10m means and standard deviations were 0.85 ± 0.06, 0.94 ± 0.02 and 0.94 ± 0.01, respectively (Figure 4). Stepwise Downscaling: Three regression steps enabled to predict 12 bands at 3 m. The first regressions downscaled S1 and S9 from 60 to 20 m, ranging from R 2 test of 0.84 to 0.99, respectively. The second round of regressions downscaled both new 20-m S1 and S9, as well as native 20-m 6 bands, ranging from R 2 test of 0.95 (S6) to 0.99 (S7, S8A, S9 and S11). Finally, the last round of regressions downscaled the 12 10-m bands, native or not, to 3 m spatial resolution, ranging from R 2 test of 0.84 (S2) to 0.99 (S1 and S8). The overall mean and standard deviation of the last round achieved R 2 test=0.95 ± 0.01, and statistics of the first and second rounds hit 0.91 ± 0.05 and 0.98 ± 0.01, respectively ( Figure 5).
The first augmentation (by 27.13%), from the 4-band to the straightforward 12-band dataset, could be driven by the multiplication by three of the number of the spectral predictors. This result is intuitive and corroborates the RS assumption: the higher the number of spectral bands, the better the discrimination will be (Collin, Planes, 2011). The second enhancement (by 18.32%, that is to say, by 45.45% compared to the first OA) might be attributed to the finer process tied to the stepwise versus the straightforward approach. Downscaling NN regressions were indeed more successful for the last round of the stepwise regressions (R 2 test=0.95 ± 0.01) than the single batch of the straightforward regressions (R 2 test=0.92 ± 0.02). Even if the process is longer (10 regressions more), it is highly recommended to implement the stepwise approach to produce a very satisfactory classification. Based on the best OA (that of the stepwise), the contribution of the individual NN-downscaled bands was quantified by referencing to the OA of the standard blue-green-red (S2-S3-S4) dataset (Figure 7). No spectral band, additional to the standard dataset, diminished the standard performance classification (OA=12.5%). S8 and S11 did not improve the standard OA, resulting in absence of gain. S12 brought then the lowest contribution to the standard OA. The poor results of those three bands could be explained by their large bandwidths compared to other bands (106, 91 and 175 nm, respectively). The narrow near-infrared S8A (21 nm bandwidth) band then provided a tangible increase of the standard OA (by 31.02%). The water vapour S9 and coastal aerosol S1 narrow bands (20 and 21 nm, respectively) drastically ameliorated the standard OA (by 46.4 and 50.82%). Finally, the three vegetation red edge bands (S6, S7 and S5) produced the highest contributions (45.1, 55.02, and 62.42%, respectively).

Figure 7.
Overall accuracies of the sole standard dataset, composed of S2-S3-S4 (blue-green-red, in blue bars), and the joint standard with other individual bands (in orange bars).

CONCLUSIONS
The fusion of the 12 S-2 bands, radiometrically corrected at the bottom-of-atmosphere, with the 4 3-m PS-2 bands was successful using a fully connected one-neuroned one-layered NN downscaling. Two approaches were assessed: one, straightforward, directly predicting S-2 bands at 3 m, irrespective of the native pixel size (that is to say 12 NN regressions); and another one, stepwise, first regressing both 60m S-2 bands at 20 m, second regressing resulting 6 20-m bands at 10 m, and third regressing 12 10-m bands at 3 m pixel size (that is to say 22 NN regressions). Both approaches were highly conclusive in a slight favour of the more precise but longer one (overall R 2 test=0.92 ± 0.02 and 0.95 ± 0.01, respectively). One-neuroned two-layered NN classifications of the 3-m scene, including 8 common coastal use and cover classes, showed that the 4-band PS-2 dataset was increasingly surpassed by the 12band straightforward and stepwise datasets (OA=48.12, 75.25 and 93.57%, respectively). Added to the blue-green-red standard dataset, the most contributing spectral bands, issued from the NN downscaling, were the vegetation red edge narrow bands, followed by the atmosphere and near-infrared narrow bands, while the wider bands barely or did not contribute to the standard OA. Even if the NN regression downscaling is more industrious, we advocate to implement the stepwise approach, especially when the goal is the supervised classification of the coastal landscape.