Underwater target classification in synthetic aperture sonar imagery using deep convolutional neural networks

In underwater synthetic aperture sonar (SAS) imagery, there is a need for accurate target recognition algorithms. Automated detection of underwater objects has many applications, not the least of which being the safe extraction of dangerous explosives. In this paper, we discuss experiments on a deep learning approach to binary classification of target and non-target SAS image tiles. Using a fused anomaly detector, the pixels in each SAS image have been narrowed down into regions of interest (ROIs), from which small target-sized tiles are extracted. This tile data set is created prior to the work done in this paper. Our objective is to carry out extensive tests on the classification accuracy of deep convolutional neural networks (CNNs) using location-based cross validation. Here we discuss the results of varying network architectures, hyperparameters, loss, and activation functions; in conjunction with an analysis of training and testing set configuration. It is also in our interest to analyze these unique network setups extensively, rather than comparing merely classification accuracy. The approach is tested on a collection of SAS imagery.

Sidescan imaging sonar performance is often described using metrics such as resolution, maximum range, and area coverage rate. These attributes focus on describing how finely and over what area a sensor can make imagery of the seafloor and targets. While these metrics are important, they are inadequate for fully characterizing the quality of sonar imagery. Shadow regions often carry as much, and sometimes even more, information than the direct return from a target. The shadow contrast within an image is therefore a critical property, and it is the result of a complex interaction between the sonar hardware, the environment, and the signal processing. This paper builds on key results from the synthetic aperture radar and sonar literature to develop a comprehensive, quantitative model for predicting the shadow contrast ratio. A model for the contrast ratio is constructed using an approach that is similar to the development of the sonar equation. This ratio describes the average relative level between the seafloor and a shadow. The model includes the effects of the transducer and array design, ambient noise, quantization noise, unwanted volume and surface reverberation, and the signal processing used to construct the synthetic aperture sonar (SAS) imagery. The shadow contrast predicted by the model is compared to the contrast measured from SAS imagery, where close agreement is observed. Examples are presented where the model is applied to a hypothetical high-frequency imaging SAS sensor in a typical operating environment. The shadow contrast ratio is estimated as a function of the sensor range and key parameters of interest, such as receiver channel spacing, sediment backscatter coefficient, and the time-bandwidth product of the transmitted signal. At long ranges, the contrast ratio is limited by additive noise interference, while at near ranges the effect of the receiver channel spacing dominates the contrast ratio through along-track ambiguities. The use of this tool for analysis and prediction of image quality has significant implications for system design, mission planning, and data processing.

The sequence of echoes detected by an active Synthetic Aperture Sonar (SAS) is coherently added in an appropriate way to produce an image with greatly enhanced resolution in the azimuth, or along-track direction when compared with an image obtained from a standard Side Looking Sonar (SLS). The SAS processing originates from the Synthetic Aperture Radar (SAR) concept. A complete introduction to SAR technique can be found in (Sherwin et al., 1962), (Walker, 1980), (Wiley, 1985) and (Curlander & McDonough, 1991). Raytheon was issued in 1969 a patent for a high-resolution seafloor imaging SAS (Walsh, 1969) and 1971 analyzed a potential system in terms of its resolution and signal-to-noise ratio. Cutrona was the first well-known radar specialist to point out how the various aspects of SAR could be translated to an underwater SAS (Cutrona, 1975). Hughes (1977) compared the performance of a standard SLS to an SAS and showed that the potential mapping rate for SAS was significantly higher than for side-looking sonar. At the time there was an assumption that the instability of the oceanic environment would prevent the formation of SAS imagery. Experimental work, which was performed by Williams (1976) and Christoff et al. (1982), refuted the instability worry. The verification of this assertion performed at higher frequencies by Gough & Hawkins (1989). Later, other concerns regarding the stability of the towed platform were also raised and some railor wire-guided trails where set up to avoid this extra complication. Nowadays there are a multiple of systems as hull mounted SAS systems, towed SAS systems and Autonomous Underwater Vehicles (AUV) systems. For further reading one can find an extended historical background of SAS in (Gough & Hawkins, 1997). Time and experience were needed to adapt SAR algorithms to SAS systems; the SAS systems use smaller radiating elements in proportion to the wavelength, which leads to higher radiation pattern of SAS with respect to SAR. The range migration effect on synthetic aperture processing is significant and pronounced in SAS imagery. An additional difference between SAR and SAS systems is the synthetic aperture time being greater in one order of magnitude in SAS, which leads to a phase corruption due to the medium fluctuations and platform instabilities. Typical synthetic aperture times for SAR are of the order of seconds with a medium coherence of some days, whereas for SAS the typical synthetic aperture time is of the order of several minutes with a similar medium coherence time (Marx et al. 2000). O pe n A cc es s D at ab as e w w w .in te ch w eb .o rg

Efficient three-dimensional (3-d) convolutional neural networks (CNNs) are designed and trained for an underwater target classification task with volumetric synthetic aperture sonar (SAS) imagery. (The third dimension of the data represents depth into the sediment, thereby enabling the consideration of buried underwater objects.) The use of tiny networks containing relatively few parameters makes training with enormous input data volumes feasible even with modest computational power and limited computer memory. The promise of the approach is demonstrated for both buried and proud man-made objects present in real, measured SAS data cubes collected at aquatic sites by an experimental volumetric sonar system, called the Sediment Volume Search Sonar (SVSS). The classification performance of each 3-d CNN exhibits marked improvement over a prescreening detection algorithm alone, and the utility of an ensemble approach is also quantified. An analysis of the effective functionality of the learned networks is provided, with this also accompanied by figures showing example trained filters as well as intermediate representations of a data volume containing unexploded ordnance (UXO). The predictions of the 3-d CNN classifiers can provide valuable guidance for the efficient allocation of resources during real-world UXO remediation operations.

Characteristics of underwater targets displayed in synthetic aperture sonar (SAS) imagery vary depending on their environmental context. Discriminative features in sea grass may differ from the features that are discriminative in sand ripple, for example. Environmentally-adaptive target detection and classification systems that take into account environmental context, therefore, have the potential for improved results. This paper presents an end-to-end environmentally-adaptive target detection system for SAS imagery that performs target recognition while accounting for environmental context. First, locations of interest are identified in the imagery using the Reed-Xiaoli (RX) detector and a Non-Gaussian detector based on the multivariate Laplace distribution. Then, the Multiple Instance Learning via Embedded Instance Selection (MILES) approach is used to identify the environmental context of the targets. Finally, target features are extracted and a set of environmentally-specific k-Nearest Neighbors (k-NN) classifiers are applied. Experiments were conducted on a collection of both high and low frequency SAS imagery with a variety of environmental contexts and results show improved classification accuracy between target classes when compared with classification results with no environmental consideration.

Phase gradient autofocus (PGA) is widely used in spotlight SAR autofocus with strong robustness. However, being the different nature of synthetic aperture sonar (SAS) imagery, traditional PGA algorithm usually fails to yield satisfactory result on SAS image without modification. Three measures are taken to ensure that PGA can be applied in stripmap SAS autofocus. A SAS image is divided into sub-images which ensures that the phase error is invariant in each sub-image; The image shift in azimuth can be avoided by removing the mean value of the phase gradient and by using strong point targets in the image; The estimation accuracy can be improved by selecting range bins according to energy criteria combined with quality criteria. The proposed method is successfully applied on SAS image and excellent results are attained.

A new, two-stage approach for performing automatic target recognition (ATR) in multi-band synthetic aperture sonar (SAS) imagery is proposed. The approach consists of a simple detection algorithm, called MondrianB, followed by a more sophisticated classification stage based on convolutional neural networks (CNNs). The approach is extensible in that the same method can be used, without any modifications, for sensor data in the form of an arbitrary number of co-registered image bands (including one). The detection stage does not require any labeled training data. The classification stage relies on ensembles of small, efficient CNNs that collectively provide robust alarm predictions by leveraging unique network architectures and multiple input-data representations. The performance of the proposed ATR approach is verified on a sizable set of data collected at sea by a three-band SAS system. Specifically, the experiments conducted quantity the utility of each imaging band on both the detection and classification stages of a man-made object-recognition task. It is shown that leveraging information from the additional bands improves both detection and classification performance.

Synthetic aperture sonar (SAS) imagery can generate high resolution images of the seafloor. Thus, segmentation algorithms can be used to partition the images into different seafloor environments. In this paper, we compare two possibilistic segmentation approaches. Possibilistic approaches allow for the ability to detect novel or outlier environments as well as well known classes. The Possibilistic Fuzzy Local Information C-Means (PFLICM) algorithm has been previously applied to segment SAS imagery. Additionally, the Possibilistic K-Nearest Neighbors (PKNN) algorithm has been used in other domains such as landmine detection and hyperspectral imagery. In this paper, we compare the segmentation performance of a semi-supervised approach using PFLICM and a supervised method using Possibilistic K-NN. We include final segmentation results on multiple SAS images and a quantitative assessment of each algorithm.

Current sonar autofocus techniques for blur removal originate in the radar community but have not provided a complete solution for Synthetic Aperture Sonar (SAS) imagery. The wide-beam, wide-band nature of SAS imagery makes implementation of Synthetic Aperture Radar (SAR) autofocus techniques difficult. This paper describes a generalisation of the standard Phase Gradient Autofocus (PGA) algorithm used in spotlight SAR that allows operation with stripmap SAS geometries. PGA uses prominent points within the target scene to estimate image blurring and phase errors. We show how PGA can be generalised to work with wide-band, wide-beam stripmap geometries. The SPGA method works by employing wave number domain 2D phase estimation techniques. The 2D phase errors are related to aperture position errors using the wave number transform. Robust sway estimates are obtained by using redundancy over a number of target points. We also present an improved Phase Curvature Autofocus (PCA) algorithm using the wave number transform. Preliminary results from the two algorithms (both on field-collected and simulated data sets) are presented and related to those obtained using previous methods. A discussion of SPGA's benefits over traditional algorithms and the limitations of the SPGA algorithm is presented. The SPGA algorithm was found to perform better than 2-D PCA on both simulated and field-collected data sets. Further testing on a variety of target scenes and imagery is required to investigate avenues of autofocus improvement.

Synthetic aperture sonar (SAS) suffers from a fundamental problem: The navigation data accuracy required for coherent summation of the echo signals is not directly achievable. Unamended this issue leads to heavily blurred images. Fortunately, under certain conditions, algorithms like displaced phase center antenna (DPCA) can be used to fine tune the navigation from the raw echo data itself. In contrary to DPCA, strip-map phase gradient autofocus (SPGA) is an algorithm which extracts residual phase errors from processed synthetic aperture images. The results can be used to correct and reprocess the raw data yielding images with reduced blurring. In the presence of overlapping phase centers in raw data, as required for data driven micro navigation like DPCA, some assumptions of SPGA are violated and can cause degradation of it's results. In this paper we investigate the effect of overlapping (redundant) phase centers (RPC) in raw data on the performance of SPGA. Results indicate that the use of redundant data during image formation can bias the phase information in SAS imagery and therefore affects SPGA focusing. In this case, side lobe levels in the focused images are increased compared to images processed without RPCs. Suitable selection schemes should be employed prior to processing to avoid biasing of the phase information due to data collected at redundant phase centers if SPGA processing on top of DPCA is required.

Underwater object classification combining SAS and transferred optical-to-SAS Imagery

A new algorithm called the Mondrian detector has been developed for object detection in high-frequency synthetic aperture sonar (SAS) imagery. If a second (low) frequency-band image is available, the algorithm can seamlessly exploit the additional information via an auxiliary prescreener test. This flexible single-band and multiband functionality fills an important capability gap. The algorithm’s overall prescreener component limits the number of potential alarms. The main module of the method then searches for areas that pass a subset of pixel-intensity tests. A new set of reliable classification features has also been developed in the process. The overall framework has been kept uncomplicated intentionally in order to facilitate performance estimation, to avoid requiring dedicated training data, and to permit delayed real-time detection at sea on an autonomous underwater vehicle. The promise of the new algorithm is demonstrated on six substantial data sets of real SAS imagery collected at various geographical sites that collectively exhibit a wide range of diverse seafloor characteristics. The results show that—as with Mondrian’s art—simplicity can be powerful.

This paper introduces a new subspace-based detection method for multichannel (high frequency and broadband) synthetic aperture sonar (SAS) imagery. An image-dependent dictionary learning method is applied to form the appropriate dictionary matrices for representing target and nontarget image snippets. The hypothesis testing is done by forming a test statistic that relies on the residual error power ratio in representing an unknown image snippet using the target and nontarget dictionary matrices. To avoid the computational bottleneck in most dictionary learning methods, a new recursive method is introduced which does not require any matrix inversion or singular value decomposition (SVD). The proposed detection method was then implemented and benchmarked against a matched subspace detection method for detecting mine-like objects. Results are then presented on two sonar imagery data sets collected in two geographically disparate locations.

Seeing through disguise: Getting to know you with a deep convolutional neural network

A synthetic aperture sonar (SAS) system has been developed by Dynamics Technology, Inc. (processing lead), and Hughes Aircraft Naval and Maritime Systems (hardware and operation). The system operates at 50 kHz and features a 3.2-m-long, 16-element linear array and a data acquisition system that records phase and amplitude information for each element. Towbody motion is estimated from information contained in the sonar data through the use of a type of long-track interferometry, aided by data from a simple motion compensation suite consisting of Doppler sonar and a single gyro. This combination compensates motion to levels correctable by standard SAR autofocus techniques modified for SAS. The system has demonstrated, at long ranges (up to 1000 m), high-resolution images (better than 20 cm), previously achievable only with high-frequency sonars at close range. The low-carrier frequency allows high-resolution imaging of the interior structures of submerged objects, recently demonstrated by SAS imagery revealing the rib and spar structure inside a wrecked PB4Y-2 airplane in Lake Washington. [Work supported by DARPA.]