MULTISPECTRAL AERIAL IMAGES TO SUPPORT BIOTOPE INFORMATION SYSTEMS FOR MIDGE INFESTATION AND BARK BEETLE MONITORING

Monitoring environmental phenomena with assistance of remotely captured data is increasingly important in environmental studies. The observation of midge infestation or bark beetles can be considered a non-direct approach to their environments or in some cases to their impact on the ecosystem. Multispectral or hyper spectral data captured by aircrafts or UAVs assist in the observation of such areas. We want to describe the technology of the sensors, the way to process the data, extract related areas and correlate those with the ground truth situation. The examples chosen are the midge infestation in the river Rhine ecosystem and the bark beetle invasion in a forest in Saxony in Germany. Special attention is put on the consequences of such extreme sprawls for the environment and the population close to these areas. 1. MIDGE INFESTATION Aedes vexans (commonly known as “Rheinschnake”) of the species Aedes is a flying insect that uses flooded areas for breeding. It is the most abundant floodwater mosquito in Europe, accumulating up to 50,000 eggs in one m2 (Kuhn 2002). The mosquitos nourish themselves with blood, affecting the living quality of the local population (Rydzanicz et al. 2011a), especially in the warm regions with high humidity. Aedes vexans is mostly spread in southern Germany, having general a warmer climate in summer than the rest of the country. The research area is located in the wide rift valley of the Rhine between Basel and Bingen. The river is an important transportation way, and several major cities are located along the stream. As many other major rivers in Central Europe the Rhine is affected by severe flooding in the winter and spring months due to glacier runoffs and prolonged rainfalls in the catchment area. Beginning of 2021 again severe flooding was reported in the area. The mainstream of the river was modified heavily by Tulla using dams and further linearization to enhance flood prevention and the development of the shipping industry. Formerly the river was in a much wider riverbed with many meanders that through this linearization got cut off the mainstream. Currently these meanders remain as all year long flooded lakes surrounded by abundant local flora and being a habitat for different types of fauna, including Aedes. In different occasions these lakes receive fresh water, e.g., strong rainfalls or floodings. When the flood level diminishes several lakes remain that are again cut off the mainstream and offer even better habitat conditions for insects like Aedes (Clarkson, Enevoldson 2021). As this species can grow quickly even the short temper of the flooding offers perfect conditions for breeding. Additionally, the area is expected to get the new habitat of invasive species from Asia, considering rising temperatures that are monitored * Corresponding author during spring and summer months and the reduction of frozen days in winter. From an ecological point of view these developments should be thoroughly observed because invasive species will eventually repress the native species if they cannot adapt quickly enough. Although these climate changes trigger evolving of more accommodated species (Rydzanicz et al. 2011b), the rapid climate changes we can experience right now can cause species to become extinct faster than we could observe and prevent. Controlling an ecosystem is a complex intervention in the environment and will cause big impacts on other species like fish, birds, and other insects, therefore the conditions on how the ecosystem works have to be observed before any human interventions, like controlling midge populations, can be made. Monitoring such events and detecting possible breeding areas of Aedes is a declared goal in the combat of midge infestation. There is a need of controlling the population of Aedes with due regard to the local ecosystem and the other occurring species of birds, fish, and insects. In terms of the climate change a lot of endemic species are endangered and in need of Aedes as prey to survive and maintain their own species (Werner et al. 2020). It is therefore necessary to combine the control and preservation of the ecosystems as environmentally as possible. Current efforts of the local population are centered on the combat of breeding, spraying certain eco-friendly pesticides around flooded zones. This tasking is organized by the local initiative KABS (Kommunale Aktionsgemeinschaft zur Bekämpfung der Schnakenplage (German Mosquito Control) that developed high expertise in the combat of Aedes vexans. So far there is no effort done in using photogrammetric or remote sensing approaches for the detection of breeding areas, which results in very time-intensive on the ground approaches. The sizes of breeding places were not well investigated enough to be a representative 1 https://www.kabsev.de/index.php (23.03.2022) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2022 XXIV ISPRS Congress (2022 edition), 6–11 June 2022, Nice, France This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-893-2022 | © Author(s) 2022. CC BY 4.0 License. 893 factor in monitoring (Clarkson, Enevoldson 2021). Considering new data sources for monitoring and mapping of possible breeding area can be a great way to improve the precision and efficiency of the midge infestation. Data Acquisition and Processing Local authorities recognized a lack of precise data, which was addressed by using a combination of RGB and NIR images. The aim was to detect areas of low currents showing higher NDVI due to algae growth and the detection of current water levels, as they change after every flood or rain event. The combination of RGB and NIR supports monitoring in a perfect way by extracting surface models, terrain models, to evaluate the water levels and generate NDVI images. CIR Orthophotos enable the integration into a Geographic Information System (GIS) Database. An aerial mission with 2cm GSD was performed in 2018 using Phase One’s iXU-RS1900 camera as Nadir and an additional achromatic iXU-RS1000 camera with 50mm lens. The 4band system mounted on GGS’s AeroStab-M fully compensated gyrostabilizer is a flexible and powerful combination that effectively covers vast areas. In combination with a flight management system (AeroTopoL) several evaluation missions were performed with a Cessna 206 over the city of Speyer. Working with a GIS several important analyses were conducted to detect flooded areas showing enforced plant growth. Flooded areas with low current were detected by analyzing the amount of chlorophyll in the water that shows existence of algae that prefer zones of slow stream current. The habitats of Aedes were detected by combining the information of Ground Truth Data, NDVI and CIR. Figure 1. The project area close to the city of Speyer Figure 2. True Color Image of flooded area (left) and false-color composite (right) revealing chlorophyll The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2022 XXIV ISPRS Congress (2022 edition), 6–11 June 2022, Nice, France This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-893-2022 | © Author(s) 2022. CC BY 4.0 License. 894 Data has been provided to both planning teams for “Strukturund Genehmigungsdirektion Süd” (Structural and Approval Directorate South) and to “Neubaugruppe Hochwasserschutz Oberrhein" (New Construction Group Flood Protection Upper Rhine) for planning and environmental issues as well as to the KABS. Figure 3. Area detected by actual data and water level shows a typical breeding area automatically detected by cluster analytics Figure 4. Comparison of True Color and NIR composite. The low chlorophyll content of the water shows there is a higher current which means the area is not a possible breeding ground for Aedes. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2022 XXIV ISPRS Congress (2022 edition), 6–11 June 2022, Nice, France This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-893-2022 | © Author(s) 2022. CC BY 4.0 License. 895 2. BARK BEETLE MONITORING The destruction bark beetles can cause have a huge impact on the health of forests for example in the Bavarian Forest (see Fig. 5) where a complete landscape was demolished by an infestation of these insects. Droughts and heat are one of the consequences of climate change and the impact on forests are undeniable. Bark beetles are a part of forest ecosystems. They lay their eggs in the bark of already damaged trees and help breaking down the tree to nourish the ground (Lausch 2015). The quite small bark beetles develop under the bark in the phloem of trees. The phloem is living tissue and transports photosynthesis products like soluble sugars. Without a functioning phloem a tree will eventually die. If the trees are already damaged the beetles can easily enter through the bark and start breeding. Healthy trees can defend themselves by producing resin if the bark is harmed. The only problem is that with climate change, more and more trees are impaired and so the beetles can break down huge tree populations and destroy forests (Jakoby et al. 2015). Affected trees have loose barks which they ultimately shed revealing certain patterns the bark beetles leave behind. The patterns are specific for different species of bark beetles. The beetles themselves are not visible from above nevertheless the beginning stages of harm on the trees can be detected in changes of the reflection patterns from the leaves. These patterns are influenced by the water and nutrient concentrations, furthermore the concentration of cellulose and chlorophyll (Lausch 2015) due to the destruction of the phloem in the bark of trees. Investigations of forest health have been made with satellite data however these images can only be obtained on cloud-free days and restricted temporal and spatial resolution (Lukeš 2021) and other methods should be looked into so that observation can be made more frequently throughout the year. Bark beetles can cause much harm in a short period of time and with rapid changes of climate and weather conditions the constant monitoring can represent a challenge. Monitoring forests for the prevention of bark beetle infestation is a serious economic, environmental and security issue in hot summers in central Europe. The correlation between wildfires, dry summers, and forest damages due to bark beetle infestation is significant. Infested trees can break uncontrollably, and dry, hot weather can cause wild fires to start, destroying landscapes and natural forests where humans normally do not interfere with. In the eastern German region Saxony, where precipitation is generally lower than in other parts of the country wide forest areas are endangered through bark beetle invasion and climatic stress (Kanand et al. 2020). For mapping areas affected by bark beetles the 5-band technology was applied using aircrafts and UAVs to receive in a test area ultrahigh-resolution data with a resolution of 5 cm for the outer and 0.5 cm for the inner test area. A camera system based on three mid format 100 MP cameras was used to capture RGB data with the first, red edge data with the second and NIR images with the third camera. All cameras were synchronized via a daisy chain solution and calibrated to process the data with one external IO. The ultra-high-resolution test area is located close to the border to Poland and is under high risk of wildfires while the core area is related to the region of Görlitz (see Fig. 6). The test area is also affected by military use which can initiate a wildfire as well. Using such 5-band images ground truth correlation guided by forest experts is needed. In this way, the data extracted is represented with high correlation areas that can be described as high-risk areas for wildfires during climatic relevant periods. As a result, more mapping was realized and monitoring these areas by UAVs for wildfires was conducted. For the core area the detection of specific forest sickness and bark beetle infestation resulted into a guideline for protection and specific monitoring applications during hot and dry weather periods. For mapping the test areas, we used the AeroSpector UAV that is capable to carry all three cameras simultaneously. Using Real Time kinematic GNSS (RTK) the IO was already very accurate, and the remaining processing was performed in a photogrammetric environment. After then, the five bands were analyzed using remote sensing tools. The use of the red edge technology was used by relative band analysis, means a wider 720 nm and a closer 760 nm band, and then recalculated the frequency of the red edge as the differential of 720-760 nm. This gives a better noise ratio on the images in contrast to a pure red edge band filter with just 40 nm width. Ground truth was needed to determine the correlation between all five bands to detect the specific forest sickness related to the bark beetle. The analytics continue, and the cluster analytics need some more investigation for further improvement. Two other missions within the next three years are planned to optimize the data source and to indicate temporal changes. The technology presented shows first ideas on combining NIR information for NDVI application with common photogrammetrically approaches in the context of environmental monitoring and biotope mapping. Considering that NDVI analysis is one of the most common approaches in Remote Sensing, the data type gathered might not be considered as innovative. Nevertheless, satellite imagery always shows the constraints of lower spatial and temporal resolution not considering the cloud cover and data availability. Combining the application of classic remote sensing tasks with the technological advantages of aerial surveys, a step towards a fully functional, integrated photogrammetric survey method is presented. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2022 XXIV ISPRS Congress (2022 edition), 6–11 June 2022, Nice, France This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-893-2022 | © Author(s) 2022. CC BY 4.0 License. 896 Fig. 5: Forest affected by bark beetles Figure 6. The inner and outer test area for red edge detection The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2022 XXIV ISPRS Congress (2022 edition), 6–11 June 2022, Nice, France This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-893-2022 | © Author(s) 2022. CC BY 4.0 License. 897


MIDGE INFESTATION
Aedes vexans (commonly known as "Rheinschnake") of the species Aedes is a flying insect that uses flooded areas for breeding. It is the most abundant floodwater mosquito in Europe, accumulating up to 50,000 eggs in one m² . The mosquitos nourish themselves with blood, affecting the living quality of the local population (Rydzanicz et al. 2011a), especially in the warm regions with high humidity. Aedes vexans is mostly spread in southern Germany, having general a warmer climate in summer than the rest of the country. The research area is located in the wide rift valley of the Rhine between Basel and Bingen. The river is an important transportation way, and several major cities are located along the stream. As many other major rivers in Central Europe the Rhine is affected by severe flooding in the winter and spring months due to glacier runoffs and prolonged rainfalls in the catchment area. Beginning of 2021 again severe flooding was reported in the area. The mainstream of the river was modified heavily by Tulla using dams and further linearization to enhance flood prevention and the development of the shipping industry. Formerly the river was in a much wider riverbed with many meanders that through this linearization got cut off the mainstream. Currently these meanders remain as all year long flooded lakes surrounded by abundant local flora and being a habitat for different types of fauna, including Aedes. In different occasions these lakes receive fresh water, e.g., strong rainfalls or floodings. When the flood level diminishes several lakes remain that are again cut off the mainstream and offer even better habitat conditions for insects like Aedes (Clarkson, Enevoldson 2021). As this species can grow quickly even the short temper of the flooding offers perfect conditions for breeding. Additionally, the area is expected to get the new habitat of invasive species from Asia, considering rising temperatures that are monitored * Corresponding author during spring and summer months and the reduction of frozen days in winter. From an ecological point of view these developments should be thoroughly observed because invasive species will eventually repress the native species if they cannot adapt quickly enough. Although these climate changes trigger evolving of more accommodated species (Rydzanicz et al. 2011b), the rapid climate changes we can experience right now can cause species to become extinct faster than we could observe and prevent. Controlling an ecosystem is a complex intervention in the environment and will cause big impacts on other species like fish, birds, and other insects, therefore the conditions on how the ecosystem works have to be observed before any human interventions, like controlling midge populations, can be made. Monitoring such events and detecting possible breeding areas of Aedes is a declared goal in the combat of midge infestation. There is a need of controlling the population of Aedes with due regard to the local ecosystem and the other occurring species of birds, fish, and insects. In terms of the climate change a lot of endemic species are endangered and in need of Aedes as prey to survive and maintain their own species (Werner et al. 2020). It is therefore necessary to combine the control and preservation of the ecosystems as environmentally as possible. Current efforts of the local population are centered on the combat of breeding, spraying certain eco-friendly pesticides around flooded zones. This tasking is organized by the local initiative KABS (Kommunale Aktionsgemeinschaft zur Bekämpfung der Schnakenplage 1 (German Mosquito Control) that developed high expertise in the combat of Aedes vexans. So far there is no effort done in using photogrammetric or remote sensing approaches for the detection of breeding areas, which results in very time-intensive on the ground approaches. The sizes of breeding places were not well investigated enough to be a representative factor in monitoring (Clarkson, Enevoldson 2021). Considering new data sources for monitoring and mapping of possible breeding area can be a great way to improve the precision and efficiency of the midge infestation.

Data Acquisition and Processing
Local authorities recognized a lack of precise data, which was addressed by using a combination of RGB and NIR images. The aim was to detect areas of low currents showing higher NDVI due to algae growth and the detection of current water levels, as they change after every flood or rain event. The combination of RGB and NIR supports monitoring in a perfect way by extracting surface models, terrain models, to evaluate the water levels and generate NDVI images. CIR Orthophotos enable the integration into a Geographic Information System (GIS) Database. An aerial mission with 2cm GSD was performed in 2018 using Phase One's iXU-RS1900 camera as Nadir and an additional achromatic iXU-RS1000 camera with 50mm lens. The 4band system mounted on GGS's AeroStab-M fully compensated gyrostabilizer is a flexible and powerful combination that effectively covers vast areas. In combination with a flight management system (AeroTopoL) several evaluation missions were performed with a Cessna 206 over the city of Speyer. Working with a GIS several important analyses were conducted to detect flooded areas showing enforced plant growth. Flooded areas with low current were detected by analyzing the amount of chlorophyll in the water that shows existence of algae that prefer zones of slow stream current. The habitats of Aedes were detected by combining the information of Ground Truth Data, NDVI and CIR. Data has been provided to both planning teams for "Strukturund Genehmigungsdirektion Süd" (Structural and Approval Directorate South) and to "Neubaugruppe Hochwasserschutz Oberrhein" (New Construction Group Flood Protection Upper Rhine) for planning and environmental issues as well as to the KABS.

BARK BEETLE MONITORING
The destruction bark beetles can cause have a huge impact on the health of forests for example in the Bavarian Forest (see Fig. 5) where a complete landscape was demolished by an infestation of these insects. Droughts and heat are one of the consequences of climate change and the impact on forests are undeniable. Bark beetles are a part of forest ecosystems. They lay their eggs in the bark of already damaged trees and help breaking down the tree to nourish the ground (Lausch 2015). The quite small bark beetles develop under the bark in the phloem of trees. The phloem is living tissue and transports photosynthesis products like soluble sugars. Without a functioning phloem a tree will eventually die. If the trees are already damaged the beetles can easily enter through the bark and start breeding. Healthy trees can defend themselves by producing resin if the bark is harmed. The only problem is that with climate change, more and more trees are impaired and so the beetles can break down huge tree populations and destroy forests (Jakoby et al. 2015). Affected trees have loose barks which they ultimately shed revealing certain patterns the bark beetles leave behind. The patterns are specific for different species of bark beetles. The beetles themselves are not visible from above nevertheless the beginning stages of harm on the trees can be detected in changes of the reflection patterns from the leaves. These patterns are influenced by the water and nutrient concentrations, furthermore the concentration of cellulose and chlorophyll (Lausch 2015) due to the destruction of the phloem in the bark of trees. Investigations of forest health have been made with satellite data however these images can only be obtained on cloud-free days and restricted temporal and spatial resolution (Lukeš 2021) and other methods should be looked into so that observation can be made more frequently throughout the year. Bark beetles can cause much harm in a short period of time and with rapid changes of climate and weather conditions the constant monitoring can represent a challenge.
Monitoring forests for the prevention of bark beetle infestation is a serious economic, environmental and security issue in hot summers in central Europe. The correlation between wildfires, dry summers, and forest damages due to bark beetle infestation is significant. Infested trees can break uncontrollably, and dry, hot weather can cause wild fires to start, destroying landscapes and natural forests where humans normally do not interfere with. In the eastern German region Saxony, where precipitation is generally lower than in other parts of the country wide forest areas are endangered through bark beetle invasion and climatic stress (Kanand et al. 2020). For mapping areas affected by bark beetles the 5-band technology was applied using aircrafts and UAVs to receive in a test area ultrahigh-resolution data with a resolution of 5 cm for the outer and 0.5 cm for the inner test area.
A camera system based on three mid format 100 MP cameras was used to capture RGB data with the first, red edge data with the second and NIR images with the third camera. All cameras were synchronized via a daisy chain solution and calibrated to process the data with one external IO.
The ultra-high-resolution test area is located close to the border to Poland and is under high risk of wildfires while the core area is related to the region of Görlitz (see Fig. 6).
The test area is also affected by military use which can initiate a wildfire as well. Using such 5-band images ground truth correlation guided by forest experts is needed. In this way, the data extracted is represented with high correlation areas that can be described as high-risk areas for wildfires during climatic relevant periods. As a result, more mapping was realized and monitoring these areas by UAVs for wildfires was conducted.
For the core area the detection of specific forest sickness and bark beetle infestation resulted into a guideline for protection and specific monitoring applications during hot and dry weather periods.
For mapping the test areas, we used the AeroSpector UAV that is capable to carry all three cameras simultaneously. Using Real Time kinematic GNSS (RTK) the IO was already very accurate, and the remaining processing was performed in a photogrammetric environment. After then, the five bands were analyzed using remote sensing tools. The use of the red edge technology was used by relative band analysis, means a wider 720 nm and a closer 760 nm band, and then recalculated the frequency of the red edge as the differential of 720-760 nm. This gives a better noise ratio on the images in contrast to a pure red edge band filter with just 40 nm width. Ground truth was needed to determine the correlation between all five bands to detect the specific forest sickness related to the bark beetle. The analytics continue, and the cluster analytics need some more investigation for further improvement. Two other missions within the next three years are planned to optimize the data source and to indicate temporal changes.
The technology presented shows first ideas on combining NIR information for NDVI application with common photogrammetrically approaches in the context of environmental monitoring and biotope mapping. Considering that NDVI analysis is one of the most common approaches in Remote Sensing, the data type gathered might not be considered as innovative. Nevertheless, satellite imagery always shows the constraints of lower spatial and temporal resolution not considering the cloud cover and data availability. Combining the application of classic remote sensing tasks with the technological advantages of aerial surveys, a step towards a fully functional, integrated photogrammetric survey method is presented. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France