Selection of Pre-training Datasets for Sonar Image Classification

Deep learning based computer vision approaches for smart agricultural applications

Rotated target recognition is a challenge for Convolutional Neural Networks (CNN), and the current solution is to make CNN rotational invariant through data augmentation. However, data augmentation makes CNN easy to overfit small scale sonar image datasets, and increases its numbers of parameters and training time. This paper proposes to recognize rotated targets of sonar images using a novel CNN with Rotated Inputs (RICNN), which doesn’t need data augmentation. During training, RICNN was trained with sonar images of targets only at one orientation, which avoid it to learn multiple rotated versions of the same targets, and reduces both number of parameters and training time of CNN. During testing, RICNN calculated classification scores for each test image and its all-possible rotated versions. The max of these classification scores were used to simultaneously estimate the category and orientation of each target. Besides, to improve the generalization of RICNN on imbalanced sonar datasets, this paper also designs an imbalanced data sampler. Experiments on a self-made small, imbalanced sonar image rotated target recognition dataset show that the improved RICNN achieves 4.25% higher classification accuracy than data augmentation, and reduces the number of parameters and training time to 2.25% and 19.2% of that of data augmentation method. Moreover, RICNN achieves comparable orientation estimation accuracy with a CNN orientation regressor trained with data augmentation. Codes, dataset are publicly available.

This paper proposes a method to detect underwater objects using sonar image simulator and convolutional neural network (CNN). Instead of simulating very realistic sonar images which is computationally complex, we implemented a simple sonar simulator that calculates only semantic information. Then, we generated training images of target objects by adding randomized degradation effects to the simulated images. The CNN trained with these generated images is robust to the degradation effects inherent in sonar images and thus can detect target objects in real sonar images. We verified the proposed method using the sonar images captured at sea through field experiments. The proposed method can implement object detection more easily because it only uses simulated images instead of real sonar images which are challenging to acquire. The proposed method can also be applied to other sonar-image-based algorithms.

At present, convolutional neural networks have achieved good results in fields such as image classification, image detection, target segmentation, target tracking, and situation estimation. The network model is trained by batch data to process the image and video rapidly and efficiently. Due to a large number of common convolutional neural network models, the actual effects of the same data set under different network models are different. Therefore, to study and select the appropriate network in the field of sonar pulse image recognition, python is applied to building convolutional neural network structures of InceptionV3, InceptionResNetV2, mobilenetV3, VGG16, DenseNet121, and NASNetMobile. In addition, pulse sonar image simulation data set is used for experiments. The results show that mobilenetV3 is the most suitable network structure for pulse sonar image recognition considering both the running speed and accuracy.

Object detection and classification in the water enhances not only the application of the autonomous underwater vehicle(AUV) but also localization of the AUV. Object detection and classification using sonar images are challenging problems due to low resolution and low signal-to-noise ratio. In this paper, we propose shape classification method using multi-view sonar images for AUV. To train multi-view of sonar images, we used network which is connected in parallel with convolutional neural network(CNN). We used Alex-net for the basic CNN model. The extracted features by the CNN are collected through the pooling layer and connected to the fully connected layer to classify the shape. To overcome the lack of training data, sonar simulator was used to generate data set. As a result, 6 shape are well classified and also shows possibility for the recognition of the real sonar images acquired in water tank.

In many medical domains recent advances in deep learning-based computer vision have moved the model performance towards (or exceeding in some cases) the level of human experts. On object classification and localization tasks, state-of-the-art classifiers based on convolutional neural networks (CNNs) achieve great accuracy. According to its rapid spread to numerous areas of the body, skin cancer is the most irritating sort of cancer diagnosis. Melanoma is a fatal kind of skin cancer that, if found early enough, can be treated with simple surgery. Therefore, Dermatologists can treat patients and save their lives by accurately classifying skin lesions in their early stages. Doctors and patients might both profit greatly from quick and precise diagnosis. As a result, computer-assisted diagnostic support systems is required. Motivated by this, this paper introduce a proposed model for melanoma classification. ISIC 2020, the most recent well-known public challenge dataset, is utilized to test the suggested model's capacity to classify Melanoma. Moreover, with less than 2% malignant samples, the dataset is significantly unbalanced. We utilise random over-sampling and data augmentation for the handling of unbalanced data. In addition, class weight technique was used to provide a weight for each class, so that the result is a classification that can equally learn from all classes, which places greater focus on the minority class. The suggested model for melanoma classification uses EfficientNet-B6 which it deep Convolutional Neural Networks (CNN) for classifying patients’ skin lesions into malignant or benign. The results of the research demonstrated that the suggested 97.84% system accuracy ratio is competitive with other models.

Deep Learning methods are currently the state-of-the-art in many Computer Vision and Image Processing problems, in particular image classification. After years of intensive investigation, a few models matured and became important tools, including Convolutional Neural Networks (CNNs), Siamese and Triplet Networks, Auto-Encoders (AEs) and Generative Adversarial Networks (GANs). The field is fast-paced and there is a lot of terminologies to catch up for those who want to adventure in Deep Learning waters. This paper has the objective to introduce the most fundamental concepts of Deep Learning for Computer Vision in particular CNNs, AEs and GANs, including architectures, inner workings and optimization. We offer an updated description of the theoretical and practical knowledge of working with those models. After that, we describe Siamese and Triplet Networks, not often covered in tutorial papers, as well as review the literature on recent and exciting topics such as visual stylization, pixel-wise prediction and video processing. Finally, we discuss the limitations of Deep Learning for Computer Vision.

In recent years Neural Networks and especially Deep Neural Networks (DNN) have seen a rise in popularity. DNNs have increased the overall performance of algorithms in applications such as image recognition and classification, 2D and 3D pose detection, natural language processing, and time series forecasting. Especially in image classification and recognition, so-called Convolutional Neural Network (CNN) have gained high interest as they can reach a high accuracy, which makes them a viable solution in cancer detection or autonomous driving. As CNNs are widely used in image tasks, different visualizations techniques have bee developed to show how their internals are working. Apart from image tasks, CNNs are applicable to other problems, e.g., time series classification, time series forecasting, or natural language processing. CNNs in those contexts behave similarly, allowing for the same visualization techniques to make them more interpretable. In this chapter, we adapt image visualization algorithms to time series problems, allowing us to build granular, intuitively interpretable feature hierarchies to make a time series forecast as understandable as an image recognition task. We do so by using our previous work on power time series forecasting using CNN Auto Encoders (AEs) and applying typical CNN visualization techniques to it. Thus, we guid computer scientists to provide better interpretable figures for a time series forecasting task to application domain experts.KeywordsVisualizationConvolutional neural networksTime series data

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

This is the first of a two-part lesson introducing deep learning based computer vision methods for humanities research. Using a dataset of historical newspaper advertisements and the fastai Python library, the lesson walks through the pipeline of training a computer vision model to perform image classification.

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.

Background. High-throughput screening of immunotherapies is crucial for identifying promising candidates against various cancer indications and accelerating the pre-clinical development of cell therapies. Two technologies have shown promise in quantifying therapy effectiveness: patient-derived organoid (PDO) models co-cultured with cell therapies, and deep learning-based computer vision, which facilitates large-scale, automated image analysis to extract quantitative metrics of treatment efficacy and cell biology effects. Here, we use deep networks to predict PDO viability from brightfield images and compute explainable features which are used to stratify a large cohort of cell therapies, tested on multiple tumoroid lines and indications. Methods. Time-lapse confocal microscopy was used to record images of a large cohort of 27 cell therapies co-cultured with 15 different patient-derived tumor organoids (PDO) lines, across 8 cancer indications (breast, colorectal, endometrial, gastric, head and neck, liver, lung, pancreas) over the course of 72 hours, and at 2 different effector-to-target concentrations. PDO viability was measured by terminal TO-PRO-3 staining. A convolutional deep network was trained to perform label-free predictions of PDO viability from brightfield images at each timepoint (N=11, 110) and evaluated on a held-out test set (N=2, 112). Deep learning segmentation models were further used to co-localize tumoroid and immune cells, and extract a set of 6 explainable features - or spatial phenotypes - which quantified 1) tumoroid area, 2) cell apoptosis intensity and 3) temporal dynamics, as well as 4) immune proliferation, 5) immune infiltration and its 6) temporal dynamics. We fitted a linear model to measure their contribution in predicting terminal viability, and used hierarchical clustering to assess associations with exposure to different therapies. Results. Our bright-field model of viability was found to be highly concordant with ground truth viability from terminal vital dye stain on a held-out test set, representing 11 TO lines across 8 distinct cancer types (Pearson’s r=0.76). A linear model of spatial phenotypes showed high correlation with terminal viability (Pearson’s r=0.70), while clustering of spatial phenotypes stratified samples into groups of non-engineered and engineered therapies with and without an additional compound. Conclusion. Our study highlights that imaging-derived metrics from brightfield imaging can quantify treatment potency and cell phenotypes in ex-vivo screens. The approach pursued here is highly scalable and permits the analysis of large scale screening experiments to assess the impact of cell therapies on PDOs. Using explainable features further allows to stratify therapies into biologically meaningful groups based on the measured phenotypic readouts. Citation Format: Luca Lonini, Madhavi Kannan, Geoffrey Schau, Stanislaw Szydlo, Vignesh Krishnaraja, Sonal Khare, Daniel Rabe, Michael Streit, Mary Flaherty, Brandon Mapes, Mark Prytyskach, Stefan Kiesgen, Gurpanna Saggu, Megan McAfee, Kate Sasser, Justin Guinney, Richard Klinghoffer, Chi-Sing Ho. Stratification of cell therapies in solid tumor organoids using deep learning-derived imaging metrics [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 7446.

Deep learning performs a great role in solving many problems, particularly with image classification. Still, the performance of many architectures is often limited at some points, especially with regular architecture. In this paper, we propose an efficient and noncomplex image classification architecture using deep learning based on the most popular algorithms: Convolutional Neural Network (CNN) for Feature extraction, Stacked AutoEncoder (SAE) for reducing the dimensionality, and Recurrent Neural Network (RNN) for increasing the accuracy. The proposed network, called Convolutional Stacked AutoEncoder Recurrent Neural Network (CSAERNet), that takes the input to be features-extracted by CNN then put as input to SAE for dimensionality reduction after that the RNN takes it as input to increase the accuracy. The proposed CSAERNet has been evaluated on two widely-used benchmark datasets; CIFAR-10 and CIFAR-100. The obtained results are compared with the latest state-of-the-art approaches, the results showed that the proposed CSAERNet perform well on both datasets and outperforms on other models.

Generative deep learning is becoming a widely used approach in geological modelling, especially in problems that involve optimization processes under uncertainty like history matching. The basic idea of reservoir modelling and history matching by generative deep learning is to train some kind of reservoir generator and provide optimization under different constraints. There are different approaches that have been developed to utilize various modifications of Generative Adversarial Networks or Variational Autoencoders to implement reservoir generators, but most of them have "conventional" convolution neural networks, hence all data have to be regular (rectangular) in structure. This should be recognized as a limitation. In this work, we introduce a novel approach of reservoir modelling with Variational Autoencoders based on graph convolutions as opposed to "conventional" convolutions. In our approach reservoir model is considered as a graph, i.e., not a structured data type. Graph convolutions can deal with these data types, and connection with Variational Autoencoders provides us with the capability to generate initially unstructured reservoirs. Variational Autoencoders (VAE) demonstrate the ability to implicitly parameterize geological representations into a latent space of reduced dimensionality and provide ways to uncertainty quantification and production profiling among various geological concepts. In the first part, we introduce the motivation to Graph Variational Autoencoders (GVAE) as opposed to conventional Generative Deep learning models. The rest of the paper is dedicated to experiments on a synthetic dataset with two different geological scenarios. We show that trained GVAE performs the generation of reservoir models with reliable geology. A latent space structure between Encoder and Decoder of GVAE is represented, and interpretation considering geology under investigation is performed. Finally, we show how latent space can help to estimate the uncertainty of production and an optimization workflow under well data constraints.

Automatic Target Recognition (ATR) in single-view sonar imagery is severely hampered by geometric distortions, acoustic shadows, and incomplete target information due to occlusions and the slant-range imaging geometry, which frequently give rise to misclassification and hinder practical underwater detection applications. To address these critical limitations, this paper proposes a Multi-View Deep Convolutional Neural Network (MVDCNN) based on feature-level fusion for robust sonar image target recognition. The MVDCNN adopts a highly modular and extensible architecture consisting of four interconnected modules: an input reshaping module that adapts multi-view images to match the input format of pre-trained backbone networks via dimension merging and channel replication; a shared-weight feature extraction module that leverages Convolutional Neural Network (CNN) or Transformer backbones (e.g., ResNet, Swin Transformer, Vision Transformer) to extract discriminative features from each view, ensuring parameter efficiency and cross-view feature consistency; a feature fusion module that aggregates complementary features (e.g., target texture and shape) across views using max-pooling to retain the most salient characteristics and suppress noisy or occluded view interference; and a lightweight classification module that maps the fused feature representations to target categories. Additionally, to mitigate the data scarcity bottleneck in sonar ATR, we design a multi-view sample augmentation method based on sonar imaging geometric principles: this method systematically combines single-view samples of the same target via the combination formula and screens valid samples within a predefined azimuth range, constructing high-quality multi-view training datasets without relying on complex generative models or massive initial labeled data. Comprehensive evaluations on the Custom Side-Scan Sonar Image Dataset (CSSID) and Nankai Sonar Image Dataset (NKSID) demonstrate the superiority of our framework over single-view baselines. Specifically, the two-view MVDCNN achieves average classification accuracies of 94.72% (CSSID) and 97.24% (NKSID), with relative improvements of 7.93% and 5.05%, respectively; the three-view MVDCNN further boosts the average accuracies to 96.60% and 98.28%. Moreover, MVDCNN substantially elevates the precision and recall of small-sample categories (e.g., Fishing net and Small propeller in NKSID), effectively alleviating the class imbalance challenge. Mechanism validation via t-Distributed Stochastic Neighbor Embedding (t-SNE) feature visualization and prediction confidence distribution analysis confirms that MVDCNN yields more separable feature representations and more confident category predictions, with stronger intra-class compactness and inter-class discrimination in the feature space. The proposed MVDCNN framework provides a robust and interpretable solution for advancing sonar ATR and offers a technical paradigm for multi-view acoustic image understanding in complex underwater environments.