Back to search

MARINFORSK-Marine ressurser og miljø

Computer vision to expand monitoring and accelerate assessment of coastal fish

Alternative title: Utvikling av maskinsyn for automatisk kameraovervåking av kystnære fiskearter

Awarded: NOK 12.2 mill.

It is now common to use underwater cameras to study and monitor coastal fish populations. Currently, human experts manually identify, size and count fish, frame by frame. This represents a bottleneck for upscaling deployment and data analysis. CoastVision will apply deep learning to develop automated detection and sizing of coastal fish caught on camera. The computer vision will also be trained to identify fish in the wild by their natural “barcodes” that distinguish species, sexes and individuals, such as differences in body shape and skin coloration patterns. Individual identification and reliable re-identification is the most innovative and novel aspect of CoastVision and will open new opportunities to study behaviour, growth and survival of fish in their natural habitat. CoastVision will focus on Atlantic cod, ballan wrasse and corkwing wrasse, all commercially important species with complex, high-contrast skin patterns. This feature will be the final step in a fully automated video analysis pipeline that will identify, track, size and count fish in video feeds from long term monitoring stations. The pipeline will be integrated into ongoing surveys and case studies whose main objective is to better understand the factors that affect the reproduction, recruitment and survival of commercially and ecologically important coastal fishes. Further, CoastVision will support studies on short- and long-term temporal dynamics of fish communities, including detecting as the arrival of invasive species, distribution shifts and altered animal behaviour associated withy climate change or other environmental stressors. Widespread adoption of camera-based monitoring with integrated computer vision will revolutionize our ability to observe, understand and respond to ecological change at scales far more refined than is currently possible.

Effective monitoring and management of coastal ecosystems is limited by observation methods. Underwater cameras are increasingly being used to monitor and study coastal fish communities; a major bottleneck for upscaling their use is dependence on human experts for image and video analysis. CoastVision will use the power of deep learning to refine and extend a computer vision pipeline for detecting, classifying and sizing the key fish species in shallow water coastal ecosystems, facilitating a transition to fully automated video analysis. Our models will be trained on data sets from several different surveys, ensuring cost-efficient development of routines that will be widely applicable. Computer vision for re-identifying (re-ID) individuals solely based on their unique visible features will also be developed. This novel aspect of CoastVision could ultimately provide new opportunities to obtain detailed knowledge about behaviour and population dynamics in wild fish populations, with minimal negative impact on animals and habitats and at a low cost. Our focal species for re-ID are Atlantic cod, ballan wrasse and corkwing wrasse, commercially important species with complex, high-contrast skin patterns. To generate the necessary training data for re-ID we will use synchronized radio frequency identification and camera systems. CoastVision’s automated video analysis pipeline will be integrated into ongoing ecosystem surveys and case studies whose main objective is to better understand the factors that affects the reproduction, recruitment and survival of commercially important coastal species. As such, CoastVision will contribute to independent, but complementary, research objectives. The project will advance the international research front for applied machine learning in marine ecology, which ultimately can revolutionize our ability to observe, understand and respond to ecological change at scales far more refined than is currently possible.

Funding scheme:

MARINFORSK-Marine ressurser og miljø