Sensor Fusion and Collision Avoidance for Autonomous Surface Vehicles

During the last 10 years driverless cars have emerged as a reality. In the air and below the sea surface, a similar development has already gone on for some time, as unmanned drones and unmanned submarines increasingly are being deployed for a variety of tasks. Recently, shipbuilders and shipping companies have also begun to seriously consider the possibility of unmanned ships. Unmanned ships can only be commercially used insofar as legislators, classification societies and the public opinion acknowledge this technology as safe. On the one hand, reducing human interference will reduce the number of mistakes caused by human operators. On the other hand, a ship must have sophisticated systems for interpreting its environment and executing actions and responses in order to safely operate on its own. Most importantly, the ship must be trusted not to collide with its surroundings. In this research project we have investigated systems for collision avoidance for unmanned ships. The fundamental premise for the Autosea project has been that the main building blocks for a collision avoidance system are a multi-target tracking system that keeps track of potential obstacles and a collision avoidance method that overrides the nominal motion control commands when necessary. The project has therefore made substantial theoretical contributions in both these fields. In multi-target tracking, the project has led to new understanding and methods for dealing with safety-critical sensor imperfections such as false alarms and misdetections. In motion control, we have developed several collision avoidance methods based on ideas from model-predictive control. These methods are capable of utilizing a very rich information picture in their decision-making. The Autosea project has all the time had a strong focus on real world validation. At the earliest possible opportunity in 2016 relevant radar data were recorded to guide the development of the tracking system. The collision avoidance systems have matured through a series of full-scale experiments involving test vessels from the consortium partners. The systems have used both radar and data from the automatic identification system (AIS) to obtain information about the location and motion of obstacles. On June 14th, 2019 the final demonstration of the Autosea project was arranged in Trondheimsfjorden. These experiments involved several vessels from Maritime Robotics, NTNU and Trondheim Havn and demonstrated the capability of the collision avoidance methods in conjunction with radar tracking to resolve nontrivial multi-vessel situations. Both these experiments as well as earlier experiments were conducted with arbitrary traffic passing through the test area. The breakthroughs in autonomous marine navigation achieved in this project are already paving the way for significant expansions of related research activities at NTNU and in the consortium partners. The half-scale autonomous ferry Milliampere is serving as a testbed for research on autonomous ships at NTNU and will use collision avoidance methods and sensor systems similar to those used in the Autosea project. These research activities are pursued in the NTNU-funded Autoferry project. In parallel with this, selected research challenges that were identified as important during the Autosea project are investigated by the Autosea consortium in a new research project called ?Autonomous ships, intentions and situational awareness? (Autosit). In particular, this includes long-term vessel prediction, radar-AIS fusion and extended object tracking. In addition to the 3 PhD candidates affiliated with the project, about 40 MSc candidates have also been associated with the project, and contributed to solving related tasks. Among these MSc projects we find motion control and collision avoidance for Milliampere and the ReVolt scale model, multi-sensor fusion using both radar, lidar, cameras and infrared cameras, digital twins, path and trajectory following and initial developments for the topics of the commencing Autosit project.

Prosjektets metoder for sensorfusjon og kollisjonsunngåelse vil bli brukt i videre forskning og produktutvikling, og de vil også bli brukt for verifikasjon og benchmarking av andre systemer. En rekke vellykkede demonstrasjoner av kollisjonsunngåelse basert på klassisk målfølgingsteori og modell-prediktiv kontroll setter premisser for videre forskning. Prosjektet har utdannet over 40 MSc -kandidater med kompetanse på maritim autonomi. Autonomi kan gjøre maritim varetransport konkurransedyktig med veitransport, og slik redusere forurensning, kødannelse og ulykker. Autonom maritim transport ofte vil kunne være betydelig mer saktegående enn bemannet maritim transport, og slik bidra til å redusere CO2-utslipp. En annen mulighet er autonom passasjertransport til sjøs. I første omgang vil dette være aktuelt som en «vannheis» over en kort og oversiktlig strekning. Etter hvert vil dette kunne revitalisere kystområder hvor det ikke er økonomisk gjennomførbart med bemannet passasjertransport.

Our vision is for the Norwegian maritime industry and researchers in collaboration with international partners to attain world-leading competence and knowledge in the design and verification of methods and systems for sensor fusion and collision avoidance for autonomous surface vehicles (ASVs). In particular, the research partners will develop and evaluate such methods and systems in compliance with the maritime anti-collision regulations (COLREGS), utilizing fusion of data from radar, AIS, IR, LIDAR, camera, IMU, GPS, etc. In addition to enabling commercial ASVs, the results can be used to enhance decision support systems for humans on manned vessels. The project will also provide a solid foundation for independent third-party verification of autonomous marine technology. The project is supported with funding and infrastructure by DNV GL, KONGSBERG and Maritime Robotics.