White caps, ice floes, sea states, weather conditions, small targets, moving objects… The variables affecting search conditions at sea are so wide ranging that such operations can often resemble the proverbial search for the needle in the haystack.
For search crew this is particularly challenging. As they seek to distinguish between potential objects of interest and environmental clutter, potentially facing difficult flying conditions of their own, time stretches and attention, inevitably, decreases. The introduction of Artificial Intelligence (AI) in airborne sensors used for maritime search missions holds great potential. It is the key to increasing chances of detection in a particularly harsh environment while lightening the load of search crew.
AI Distinction
“There are currently two main types of AI in use,” says Mark Palmer, Chief Technology Officer at Sentient. “The original computer vision AI looks at what is called Moving Target Indication (MTI), whereas more recent systems run a type of AI known as Deep Learning (DL).” While the two are not mutually exclusive, understanding the distinction between MTI and DL in the context of maritime search missions is critical.
MTI, as its name indicates, is a type of AI trained to automatically recognise everything that moves against a given background. It is particularly useful to identify moving objects on land, discriminating a target of interest against the clutter – such as trees, houses, stationary cars, etc. In the maritime domain, however, MTI quickly reaches its limits. Discriminating a target against a background that constantly moves requires very complex algorithms.
“That is why today most companies seeking to develop AI-enabled search systems and sensors focus their attention on DL,” Palmer notes. DL focuses on teaching a system to recognise specific objects against any given background. It relies on data gathered during trials in different conditions to constantly refine the search. The issue with the use of such system in the maritime environment, however, is that the surrounding clutter can very easily look like a target of interest. For the operator searching for a small target over a wide area, sifting through false positives can be time consuming and may lead to missing the actual object/person of interest.
Sensing Trouble
With ViDAR, Sentient has chosen to leverage the opportunities offered by both types of AI in order to provide maritime search operators with uniform, persistent sensor coverage. “The most significant difference with ViDAR is that MTI identifies everything that moves, then through DL it is programmed to ignore what the operator will never be interested in for that specific mission (e.g. white caps, ice floes, etc),” Palmer adds.
Just as importantly, Sentient has been working with AI – first MTI for its Kestrel software and now MTI with DL for ViDAR – for the past 20 years, storing large quantities of data from search missions in varying conditions and altitudes. Such wealth of data means Sentient can train ViDAR’s AI to search and recognise targets such as a small rubber boat, vessel or person up to sea state 6.
ViDAR’s AI is complemented by a Wide Area Maritime Detection (WAMI) system, offering critical wide and persistent search to countries with large maritime domains to patrol and/or search. Compared to other sensors that point and search a specific and limited area, ViDAR features a multispectral camera array to offer both persistent 180° field of view and uniform coverage. “If the system detects something, it will automatically cross-cue to the aircraft primary EO/IR turret for identification and classification, simultaneously continuing the search for other targets,” says Mick Sheehy, Special Missions Operations Support Manager at Sentient.
Finally, ViDAR is an Electro-Optical/Infra Red (EO/IR) sensor, which means it is passive and does not emit. While this may not be relevant for Search and Rescue (SAR) missions, it is crucial for Intelligence, Surveillance and Reconnaissance (ISR) as well as maritime patrol missions. Whether crew are patrolling territorial waters in search of narco-traffickers, human traffickers, or illegal fishing, they need to make sure they are not detected in the process of doing so, which would alert targets.
Pod Casting
“Another very important aspect of the ViDAR is its versatility,” notes Brent Bergan, Business Development and Government Relations at Sentient. Adaptable and scalable, depending on customer needs, it can be fitted on a wide range of platforms: from Group 1 Unmanned Aerial Systems [UAS] to bigger manned and unmanned, fixed and rotary wings aircraft. “We integrate it into the airframe of the platform and into its mission system,” adds Bergan, so that all the information can display into one single screen to facilitate the maritime operating picture and decision-making.
Yet not all customers may wish to have one sensor dedicated to one platform; this would mean that when an aircraft is in maintenance or repair, that dedicated sensor cannot be used by another capability.
As such, in March 2021 Sentient launched the ViDAR Maritime Surveillance (VMS) pod. Featuring a low Size Weight and Power (SWAP) and both EO/IR, it can be fitted to customer needs: only EO or both EO/IR, and with as many cameras as needed for the types of missions envisaged. “It is adaptable as a plug and play solution across a range of platforms such as rotary, fixed wings and UAVs of all sizes,” says Sheehy. This solution is particularly interesting for customers seeking to be able to swap the pod between platforms.
Looking Forward
“In the coming years we will continuously be introducing new capabilities in order to extend the operational envelope of the system,” says Palmer. This will include high gain cameras that will significantly improve night vision, “so that, night or day, the solution remains completely passive while providing high quality imagery and detection,” Palmer concludes. This will apply to both the maritime and the land version of ViDAR to come.