Martime AI-NAV (Artificial Intelligence / Machine Learning Sensor Fusion for Autonomous Vessel Navigation)
November 16, 2020: A day of ferry trips between Finland and Estonia became some of the best documented voyages in maritime history. Cameras, sensors, radio and satellite navigation receivers and even microphones recorded every instant of the crossings over the Baltic, gathering raw data for a new ESA-led project applying AI to the situational awareness of shipping – as an important step to full autonomy. 1)
Figure 1: Photo of the Tallink Megastar ferry (photo: AS Tallink Grupp)
The testing was overseen by a team from the Finnish Geospatial Research Institute (FGI) for an ESA project called Artificial Intelligence / Machine Learning Sensor Fusion for Autonomous Vessel Navigation, or Maritime AI-NAV for short.
“Our aim is to show how AI and machine learning can be applied to achieve autonomous situational awareness, so that a ship can reliably sense its own environment,” notes Sarang Thombre of FGI.
Figure 2: A day of ferry trips between Finland and Estonia became some of the best documented voyages in maritime history. Cameras, sensors, radio and satellite navigation receivers and even microphones recorded every instant of the crossings over the Baltic, gathering raw data for a new ESA-led project applying AI to the situational awareness of shipping – as an important step to full autonomy (image credit: FGI)
“Such autonomous systems would initially be deployed in support of human crews, for enhanced safety and efficiency – with crewless ships a much longer-term goal.
“The most experienced human ship captains will have the least trust in any single navigational device but will rather continuously cross reference between them. Similarly, our autonomous functionality will not be overly reliant on a single data source but combine and verify data from multiple sensors.
“Having gathered many gigabytes of data during this 20 August field campaign, we are now using it to train and test our data-fusing algorithms. A follow-up seagoing test will then verify their performance in practice.”
Figure 3: Photo of an installed Satnav antenna (photo credit: FGI)
The Maritime AI-NAV team plans in employing a variety of sensor types, including satellite navigation receivers – also making use of Europe’s Galileo system – inertial measurement units, monocular and stereo cameras, standard radar, ‘laser radar’ lidar and an array of microphones, along with Automatic Identification System radio signals. These AIS signals transmit position, size and routing information of all vessels above a certain class, as well as fixed infrastructure such as oil rigs or wind turbines.
Dr Thombre adds: “Satellite navigation lets the ship know where it is in the sea, while the other sensors let it know what is around it, which is essential for identifying and avoiding any obstacles.
“The different data sources operate across a variety of ranges – so radar and AIS provide longer range detection out to the horizon, while cameras and lidars come into their own at shorter distances. Plus we had a trio of microphones aboard the Megastar, determining the angle of arrival of sound from other ships. The challenge now is to fully integrate all these sources using AI and machine learning, to build up a holistic picture.”
Maritime AI-NAV is supported through ESA’s Navigation Innovation and Support Program (NAVISP), working with European industry and academia to develop innovative navigation technology.
FGI is joined in the Maritime AI-NAV consortium by Aalto University’s Sensor Informatics and Medical Technology group and maritime IT startup Fleetrange Ltd.
Figure 4: Gathering data from ferry trips (image credit: FGI)
Figure 5: Storing sensor data (photo credit: FGI)
Table 1: Artificial Intelligence / Machine Learning Sensor Fusion for autonomous Vessel Navigation 2)
1) ”Baltic ferry gathers data for self-aware sailing,” ESA Applications, 16 November 2020, URL: https://www.esa.int/Applications/Navigation/Baltic_ferry_gathers_data_for_self-aware_sailing
The information compiled and edited in this article was provided by Herbert J. Kramer from his documentation of: ”Observation of the Earth and Its Environment: Survey of Missions and Sensors” (Springer Verlag) as well as many other sources after the publication of the 4th edition in 2002. - Comments and corrections to this article are always welcome for further updates (firstname.lastname@example.org).