Cost-effective observation methodology to assess seal population sizes using unmanned aerial vehicle and automatic image analysis

This project aims to explore the use of Unmanned Aerial Vehicles (UAVs) operated from a vessel to perform aerial photo surveys of seal whelping patches. An additional goal is to develop corresponding methods to improve population assessments of harbour and grey seals (coastal seals). A second main goal is to develop new methods to analyze aerial images to be able to count seal pups in an easier way. Ice breeding seals In March 2014 and 2015 we carried put to surveys using ice-going vessels to the Greenland Sea. We tested two UAVs - Cryowing Mk.1 (CW1) and Cryowing Micro (CWM) - to perform aerial photo surveys of harp and hooded seals on the drift ice. We aimed to explore various operational challenges such as operating the UAVs in extreme cold and windy conditions, landing the smaller CWM on the helicopter platform and the larger CW1 on ice floes. Digital cameras and an IR-camera were used, and various photo altitudes and camera settings were explored. Both operations of UAVs and cameras were promising. However, in 2015 we lost the large drone (CW1) due to an unknown technical error during the first flight. The plan was to test the drone to perform longer flights (3-4 hours). The smaller drone (CWM) was successfully tested, including landings on ice floes. Coastal seals In August and November 2016, we tested an electric helicopter drone (4 propellers) operated from a smaller boat to conduct photo-surveys of harbour seals resting on skerries on the Norwegian coast. The test was so successful that we implemented the method for harbour seal counts in Vestfold and Telemark counties. We used the helicopter-drone six times during one day and covered most of the harbour seal haul-out sites in the two counties. The results, number of seals, were available after the images taken were examined on a computer a few minutes after each flight. Detection of seal pups using deep learning Automatic detection of ice breeding seal pups (harp and hooded) from aerial photos is a challenging problem due to many interfering factors, including shadow casts, patches of water and shallow ice between the ice sheets, traces of the blood caused by the birth process, and the presence of adult seals. Moreover, for the harp seals, the pup has a white fur. In order to detect the seal pups, Norwegian Computing Center in collaboration with the Institute of Marine Research have developed a detection methodology based on deep learning. Deep learning has revolutionized computer vision the last four years in terms of its ability to extract content and information from images. Central in the proposed method is a deep neural network that consists of several hundred thousands of parameters that are being learned in a so-called training process. In order to achieve this we have used an extensive amount of images of seal pups, but also typical background In the UAVSEAL project we have used about four hundred thousand training images. The technique is computationally demanding and is implemented on a graphical processing unit. Initial tests of the proposed deep learning based seal detection scheme show that we are able detect seals with a very high accuracy. By evaluating the proposed method on validation dataset, we obtained an accuracy of 99.7%. However, since the seal pups only cover a small fraction of the sea ice, the vast majority of the tested locations contain background. Thus, we will experience a significant number of false detections, even with an accuracy of 99.7%. A semi-automatic approach is therefore implemented, where a reader may easily and efficiently evaluate the detected seal pups and modify the results if necessary. Since the amount of available images of coastal seals are substantially less than for ice breeding seal, it has not been possible to learn similar deep learning networks for coastal seal detection.

The abundance estimation of ice breeding seals (harp and hooded) and coastal seals (grey and harbour) are based on aerial photographic surveys and manual inspection of aerial photographs. Photo surveys in remote areas (the West Ice) are expensive and the logistics have become increasingly difficult during the last years. The project aims to explore the use of unmanned aerial vehicle (UAV) operated from a survey vessel to photograph harp and hooded seal whelping areas. The photos will be analysed and model led using a mosaic method, which will give a total photographic coverage of the area and provides us with the opportunity to obtain a total number of seals in the covered area. Then we will explore how various sampling techniques, such as random sampling and traditional aerial strip transect methods with different transect widths and various spatial coverage along transects will affect the quality of the estimates of the seal abundance. Manual analysis of the images is extremely time consuming and costly, and involves subjective human interpretation by trained experts. This project aims at developing methodology for automating the process of counting seals from aerial images. This will be achieved through the development of new image analysis and pattern recognition techniques tailored to detect seals in digital colour images. The project will also investigate the potential of thermal infrared sensors that extract temperature characteristics of the imaged objects. The result of the research will be knowle dge about how to apply an UAV to estimate the seal population sizes, and a set of algorithms for automatic and semi-automatic detection of seals in aerial images, leading to cost-effective abundance estimation of seal populations. This will further lead t o better long-term management of seal populations. This project would make a valuable contribution by simplifying the logistics and reducing the cost of collecting field data.