Aural based scene understanding for sonar applications

The relevance is determined by the key problem of modern intelligent systems — the limited effectiveness of existing computer vision models for predicting dynamic processes in changing conditions. The need to develop high-tech technologies capable of analyzing multidimensional time series and ensuring high reliability of forecasts is a fundamental challenge in the field of artificial intelligence. The aim of the research is to develop methodological foundations for creating predictive computer vision models based on hybrid deep learning architectures for time series analysis of multimodal data. Methods. Hybrid deep learning architectures, including combinations of convolutional (CNN) and recurrent neural networks (LSTM), as well as ConvLSTM models, have been investigated. A mathematical apparatus has been developed to formalize learning processes, including specialized loss functions and optimization algorithms adapted for forecasting tasks based on time series of visual data. Results. It has been experimentally shown that the CNN-LSTM hybrid model predicts the dynamics of the development of objects 8 time steps ahead with an average absolute error of 0.069 and a coefficient of determination of 0.9127. A statistically significant superiority of this architecture over isolated convolutional or recurrent networks has been established. The effectiveness of the LSTM architecture for predicting anomalous events based on multidimensional data with an accuracy of 0.92 has been proven. The prospects of using multimodal and graph neural network models to increase the reliability of forecasts by taking into account spatial and temporal relationships are revealed. Practical significance. The results of the work form the basis for the creation of high-tech information and analytical systems of predictive analytics, applicable in various subject areas where the analysis of visual information with a timestamp is required. Discussion. The developed approaches demonstrate the potential for creating intelligent anomaly detection and decision support systems. The development of methods that are resistant to a limited amount of training data and the creation of approaches to quantify forecast uncertainty in complex dynamic environments are identified as promising areas.

The convolutional neural network (CNN) is a widely recognized model in computer vision due to its capability to utilize data correlation effectively. However, CNNs often face challenges when training on large datasets or dimensions. To address this, the Quantum Convolutional Neural Network (QCNN) introduces an innovative solution for CNN-based problems, specifically within quantum computing frameworks. By integrating CNN architecture into quantum environments, QCNN enhances the model's overall efficiency and functionality. This approach marks an initial effort to propose a solution for classification challenges in quantum physics and chemistry. Furthermore, the study highlights a computational method utilizing the Multi-scale Entanglement Renormalization Ansatz (MERA) with logarithmic depth O(log(n)). Experiments conducted using the TensorFlow Quantum platform on the MNIST dataset demonstrate that QCNNs can achieve faster learning compared to conventional CNNs.

In this work, we explore the use of Convolutional Neural Networks to forecast discretized rainfall intensity by transforming multivariate time series data into image representations using Recurrence Plots (RP). A Convolutional Neural Network (CNN) architecture which takes these image representations in order to produce classification forecasts is proposed. Experimental results in classifying DOST-PAGASA' `s color-coded rainfall advisory yield 96.27% 10-fold cross validation accuracy with a configuration of 24-hour lag and 6-hour forecast horizon.

Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Share Icon Share Twitter Facebook Reddit LinkedIn Tools Icon Tools Reprints and Permissions Cite Icon Cite Search Site Citation Matthew Sjarah, Winda Astuti, Lie Zener, Ari Fadli; Development of automatic rice plant pest detection system based on convolutional neural network. AIP Conf. Proc. 21 February 2023; 2482 (1): 030004. https://doi.org/10.1063/5.0110445 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAIP Publishing PortfolioAIP Conference Proceedings Search Advanced Search |Citation Search

Squint multi-receiver synthetic aperture sonar (SAS) data is more challenging to process than data about its common side-looking counterpart because the geometry is complicated. While traditional multi-receiver algorithms mainly handle general side-looking cases, they do not work well for squint cases. Up to now, no multi-receiver algorithm for handling squint multi- receiver SAS data has been reported. This paper focuses on an imaging algorithm for the squint multi-receiver SAS. Firstly, the “non-stop-go-stop” signal model is established for squint multi- receiver SAS. Then a squint multi-receiver nonlinear chirp scaling algorithm (SMANLCSA) is proposed that is suited to process multi-receiver SAS data with small squint angle and SMANLCSA involves three key steps. The first step is to turn squint multi-receiver data into squint single-receiver data. The second step is a Doppler centroid frequency shift. The third step is range cell migration correction using nonlinear chirp scaling. Simulations have shown that SMANLCSA can handle squint multi-receiver data more accurately and effectively than the previous algorithm..

The rich content of synthetic aperture sonar (SAS) data is typically used to generate imagery with resolution as high as theoretically possible. But when the ultimate purpose of the imagery is for detecting objects with sizes large compared to the resolution cell, exploiting the raw data in alternative ways can be more useful. We first show how multiple lower-resolution SAS images (i.e., “looks”) can be obtained in an efficient, principled manner by band-limiting the image wavenumber spectrum of a full-resolution SAS image. By combining these multiple looks (from different aspect and frequency bands), a despeckled image that enjoys greater (target) signal to (seabed) reverberation ratio can be produced. On a large set of real SAS data collected at sea, it is demonstrated how better target detection performance of underwater mines can be achieved by using the lower-resolution despeckled imagery rather than single-look, maximum-resolution imagery.

We propose a practical Convolution Neural Network (CNN) model termed the CNN for Semantic Segmentation for driver Assistance system (CSSA). It is a novel semantic segmentation model for probabilistic pixel-wise segmentation, which is able to predict pixel-wise class labels of a given input image. Recently, scene understanding has turned out to be one of the emerging areas of research, and pixel-wise semantic segmentation is a key tool for visual scene understanding. Among future intelligent systems, the Advanced Driver Assistance System (ADAS) is one of the most favorite research topic. The CSSA is a road scene understanding CNN that could be a useful constituent of the ADAS toolkit. The proposed CNN network is an encoder-decoder model, which is built on convolutional encoder layers adopted from the Visual Geometry Group’s VGG-16 net, whereas the decoder is inspired by segmentation network (SegNet). The proposed architecture mitigates the limitations of the existing methods based on state-of-the-art encoder-decoder design. The encoder performs convolution, while the decoder is responsible for deconvolution and un-pooling/up-sampling to predict pixel-wise class labels. The key idea is to apply the up-sampling decoder network, which maps the low-resolution encoder feature maps. This architecture substantially reduces the number of trainable parameters and reuses the encoder’s pooling indices to up-sample to map pixel-wise classification and segmentation. We have experimented with different activation functions, pooling methods, dropout units and architectures to design an efficient CNN architecture. The proposed network offers a significant improvement in performance in segmentation results while reducing the number of trainable parameters. Moreover, there is a considerable improvement in performance in comparison to the benchmark results over PASCAL VOC-12 and the CamVid.

HighlightsWhat are the main findings?Proposed an innovative two-channel hybrid model by integrating convolutional neural networks (CNNs) with a text-based classifier.Leveraged an extended dataset derived from multiple object recognition models, increasing input data diversity and achieving a text-based classifier accuracy of 73.30%. Achieved a significant improvement of 8.33% in accuracy compared to CNN-only models, with the hybrid model attaining an accuracy of 90.46%.What is the implication of the main finding?Efficient and Scalable Methodology: Utilized EfficientNet for CNN-based feature extraction and Bag-of-Words for text representation, ensuring computational efficiency and scalability.Application Potential: Addressed challenges in indoor scene recognition, such as complex backgrounds and object diversity, demonstrating significant potential for applications in robotics, intelligent surveillance, and assistive systems.Indoor scene recognition is a computer vision task that identifies various indoor environments, such as offices, libraries, kitchens, and restaurants. This research area is particularly significant for applications in robotics, security, and assistance for individuals with disabilities, as it enables the categorization of spaces and the provision of contextual information. Convolutional Neural Networks (CNNs) are commonly employed in this field. While CNNs perform well in outdoor scene recognition by focusing on global features such as mountains and skies, they often struggle with indoor scenes, where local features like furniture and objects are more critical. In this study, the “MIT 67 Indoor Scene” dataset is used to extract and combine features from both a CNN and a text-based model utilizing object recognition outputs, resulting in a two-channel hybrid model. The experimental results demonstrate that this hybrid approach, which integrates natural language processing and image processing techniques, improves the test accuracy of the image processing model by 8.3%, achieving a notable success rate. Furthermore, this study offers contributions to new application areas in remote sensing, particularly in indoor scene understanding and indoor mapping.

Wheat rust is a disease affecting many productive wheat regions. It is caused by a pathogenic fungi of the order Pucciniales that attacks both the stem and the leaves of the wheat crop. It is characterized by the presence of yellow-spotted patterns. If not treated, rust can cause a drop in wheat production of up to 90%. In the proposed methodology, a Deep Convolutional Neural Network (DCNN) has been trained to learn to identify two types of rust disease with the help of transfer learning. Transfer learning enables the utilization of various computer vision models that have been trained on massive datasets. Different DCNN models have been applied, such as AlexNet, VGG-16, VGG-19, ResNet-18, ResNet-34, ResNet-50, ResNet-101 and ResNet-152; which have been pre-trained on massive image datasets. The DCNN was evaluated on a set of 415 images. For all models, various performance metrics (such as accuracy, precision, recall, specificity, MCC, Cohen’s Kappa, FPR) were analyzed and ResNet-152 was found to give the best overall results. The highest accuracy achieved was 90.36%. The early identification of wheat rust diseases using this method could be used as an early warning system.

Computer vision systems are increasingly expanding their application in visual data analysis. Model training methods are undergoing the greatest development and improvement as the results of this stage significantly affect the final classification of objects and the interpretation of input information. Typically, computer vision systems use convolutional neural networks for training (Convolution Neural Network, CNN). The disadvantages of such systems are significant limitations in cross-modal learning, multimodality implementation, labeling of large amounts of data, etc. One of the ways to overcome these problems is to use Vision Transformers (ViT), which, compared to classical CNNs, have higher performance due to reduced inductive biases and high parallel computing efficiency. Introducing Self-Supervised Learning (SSL) technologies can significantly reduce the dependence on manually labeled data, contributing to the formation of generalized representations of images. Cross-Modal Learning (CML) expands the possibilities of processing them by combining data of different types. The development of the new approach, combined with the capabilities of cross-modal learning and self-learning in ViT in a single architecture, will ensure adaptability, efficiency, and system scalability in various applications. The research aims to provide a detailed overview of ViTs, approaches to their architecture, and methods for improving their efficiency. The mathematical foundations of the key concepts of ViT, cross-modal learning and self-learning, the main modifications of ViT, and their integration with SSL and CML technologies are considered. A comparison of methods using characteristics, performance, and efficiency is provided. The key challenges and prospects facing researchers and developers while creating universal models in computer vision are outlined. ViTs change computer vision by capturing global dependencies on images. Despite some challenges, ViTs provide excellent scalability and performance for large datasets. The active search for methods to overcome their limitations makes ViTs a key tool for improving image classification, object detection, and other computer vision tasks.

In existing convolutional neural networks (CNNs), both convolution and pooling are locally performed for image regions separately, no contextual dependencies between different image regions have been taken into consideration. Such dependencies represent useful spatial structure information in images. Whereas recurrent neural networks (RNNs) are designed for learning contextual dependencies among sequential data by using the recurrent (feedback) connections. In this work, we propose the convolutional recurrent neural network (C-RNN), which learns the spatial dependencies between image regions to enhance the discriminative power of image representation. The C-RNN is trained in an end-to-end manner from raw pixel images. CNN layers are firstly processed to generate middle level features. RNN layer is then learned to encode spatial dependencies. The C-RNN can learn better image representation, especially for images with obvious spatial contextual dependencies. Our method achieves competitive performance on ILSVRC 2012, SUN 397, and MIT indoor.

Object detection has become the cornerstone of various real-world applications, ranging from autonomous driving where it helps detect road objects to medical imaging for early disease prediction and gesture recognition systems. Due to their ability to operate continuously and cost-effectively, computer vision models are increasingly being employed in surveillance and monitoring tasks. Unlike humans, who can intuitively recognize and understand objects within images, machines require advanced algorithms to mimic such capabilities. A key challenge in computer vision lies in recognizing and tracking objects in real-time with accuracy and reliability. Human visual systems perform these tasks naturally and swiftly, enabling complex activities like driving. For computers to match such performance, efficient object detection models along with additional hardware such as sensors are essential. This study presents a comparative analysis of two popular deep learning-based object detection algorithms You Only Look Once (YOLO) and Faster Region-based Convolution Neural Network (Faster RCNN). Both models are evaluated based on their detection accuracy, speed, and performance in real-time scenarios. The paper aims to highlight the strengths and limitations of each model, offering insights into their suitability for different applications. The findings suggest that while both algorithms have their merits, the choice between them depends on the specific requirements of a task, such as the trade-off between detection speed and precision. Comparative study shows that Faster RCNN outperforms its accuracy compared with YOLO.

How do we understand the complex patterns of neural responses that underlie scene understanding? Studies of the network of brain regions held to be scene-selective—the parahippocampal/lingual region (PPA), the retrosplenial complex (RSC), and the occipital place area (TOS)—have typically focused on single visual dimensions (e.g., size), rather than the high-dimensional feature space in which scenes are likely to be neurally represented. Here we leverage well-specified artificial vision systems to explicate a more complex understanding of how scenes are encoded in this functional network. We correlated similarity matrices within three different scene-spaces arising from: (1) BOLD activity in scene-selective brain regions; (2) behavioral measured judgments of visually-perceived scene similarity; and (3) several different computer vision models. These correlations revealed: (1) models that relied on mid- and high-level scene attributes showed the highest correlations with the patterns of neural activity within the scene-selective network; (2) NEIL and SUN—the models that best accounted for the patterns obtained from PPA and TOS—were different from the GIST model that best accounted for the pattern obtained from RSC; (3) The best performing models outperformed behaviorally-measured judgments of scene similarity in accounting for neural data. One computer vision method—NEIL (“Never-Ending-Image-Learner”), which incorporates visual features learned as statistical regularities across web-scale numbers of scenes—showed significant correlations with neural activity in all three scene-selective regions and was one of the two models best able to account for variance in the PPA and TOS. We suggest that these results are a promising first step in explicating more fine-grained models of neural scene understanding, including developing a clearer picture of the division of labor among the components of the functional scene-selective brain network.

Script identification for scene text images is a challenging task. This paper describes a novel deep neural network structure that efficiently identifies scripts of images. In our design, we exploit two important factors, namely the image representation, and the spatial dependencies within text lines. To this end, we bring together a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN) into one end-to-end trainable network. The former generates rich image representations, while the latter effectively analyzes long-term spatial dependencies. Besides, on top of the structure, we adopt an average pooling structure in order to deal with input images of arbitrary sizes. Experiments on several datasets, including SIW-13 and CVSI2015, demonstrate that our approach achieves superior performance, compared with previous approaches.

Early detection and forecast of extreme weather events such as tropical cyclones and wildfires are becoming increasingly crucial for mitigating their catastrophic damages. Remotely-sensed satellite data enable us to detect and assimilate these events into weather models to improve their global-scale forecasts. Despite the significant advances in satellite imagery and image processing techniques, weather data assimilation methods currently employ conventional approaches that can be slow and inaccurate. This letter aims to bridge this gap by utilizing deep convolutional neural networks (CNN) to accurately and rapidly detect weather patterns in satellite images. InceptionV3, ResNet50, VGG16, and VGG19, as four state-of-the-art CNN models, are used to classify 9081 optical satellite images into five different weather events of convective cell clouds, tropical cyclones, convective roll clouds, wildfires, and dust storms. All these models obtained a satisfactory average accuracy of more than 80%. Among the CNNs, VGG-Nets were found to have difficulties in distinguishing roll convection from cell convection patterns. Overall, InceptionV3 was the best model by achieving an average accuracy of 92% in detecting such weather systems. The letter demonstrates the strength of these CNN techniques for weather pattern recognition that can be instrumental for atmospheric science research and weather/climate models.