Deep learning for source localization and seabed classification under dynamic oceanographic environments using explosive sound signals

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Deep learning (DL) is now generally acknowledged as the benchmark and evolution in machine learning (ML) fields. Further more, it has steadily suited the most extensively in computational techniques for ML, delivering excellent outcomes in several challenging intellectual works that have equal or still outperformed human ability. DL has the advantage of learning from enormous amounts of data. Here, the DL approach has been applied to classify thunderstorms and non-thunderstorms with a meteorological dataset. Daily observational hourly data sets from 2016 to 2021 have been used to classify the incidence of thunderstorms and non-thunderstorms. Three DL approaches are used: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Long Short-Term Memory (LSTM) to classify thunderstorm incidences. These DL techniques are compared with conventional ML techniques, showing that DL approaches outperformed ML. The CNN method provides the highest performance with 92.01% accuracy among all DL and ML approaches.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

Recent advancements in deep learning, particularly convolutional networks, have rapidly become the preferred methodology for analyzing medical images, facilitating tasks like disease segmentation, classification, and pattern quantification. Central to these advancements is the capacity to leverage hierarchical feature representations acquired solely from data. This comprehensive review meticulously examines a variety of deep learning techniques applied across diverse healthcare domains, delving into the intricate realm of medical imaging to unveil concealed patterns through strategic deep learning methodologies. Encompassing a range of diseases, including Alzheimer’s, breast cancer, brain tumors, glaucoma, heart murmurs, retinal microaneurysms, colorectal liver metastases, and more, the analysis emphasizes contributions succinctly summarized in a tabular form.The table provides an overview of various deep learning approaches applied to different diseases, incorporating methodologies, datasets, and outcomes for each condition.Notably, performance metrics such as accuracy, specificity, sensitivity, and other crucial measures underscore the achieved results. Specifically, an in-depth discussion is conducted on the Convolutional Neural Network (CNN) owing to its widespread adoption as a paramount tool in computer vision tasks. Moreover, an exhaustive exploration encompasses deep learning classification approaches, procedural aspects of medical image processing, as well as a thorough examination of key features and characteristics. At the end, we delve into a range of research challenges and put forth potential avenues for future improvements in the field.

In recent years, domain generation algorithms (DGAs) are the foundational mechanisms for many malware families. Mainly, due to the fact that DGA can generate immense number of pseudo random domain names to associate to a command and control (C2) infrastructures. This paper focuses on to detect and classify the pseudo random domain names without relying on the feature engineering or any other linguistic, contextual or semantics and statistical information by adopting deep learning approaches. A deep learning approach is a complex model of traditional machine learning mechanism that has received renewed interest by solving the long-standing tasks in artificial intelligence (AI) related to the field of natural language processing, image recognition, speech processing and many others. They have immense capability to extract optimal feature representations by taking input as in the form of raw input texts. To leverage this and to transfer the performance enhancement in aforementioned areas towards characterize, detect and classify the DGA generated domain names to a specific malware family, this paper adopts deep learning mechanisms with a known one million benign domain names from Alexa, OpenDNS and a corpus of malicious domain names generated from 17 DGA malware families in real time for training in character and bigram level and a trained model has been evaluated on the OSNIT data set in real-time. Specifically, to understand the effectiveness of various deep learning mechanisms, we used recurrent neural network (RNN), identity-recurrent neural network (I-RNN), long short-term memory (LSTM), convolution neural network (CNN), and convolutional neural network-long short-term memory (CNN-LSTM) architectures. Additionally, to find out an optimal architecture, experiments are done with various configurations of network parameters and network structures. All experiments run up to 1000 epochs with a learning rate set in the range [0.01-0.5]. Overall, deep learning approaches, particularly family of recurrent neural network and a hybrid network (where the first layer is CNN and a subsequent layer is LSTM) have showed significant performance with a highest detection rate 0.9945 and 0.9879 respectively. The main reason is deep learning approaches have inherent mechanisms to capture hierarchical feature extraction and long range-dependencies in sequence inputs.

More than 80% of video surveillance systems are used for monitoring people. Old human detection algorithms, based on background and foreground modelling, could not even deal with a group of people, to say nothing of a crowd. Recent robust and highly effective pedestrian detection algorithms are a new milestone of video surveillance systems. Based on modern approaches in deep learning, these algorithms produce very discriminative features that can be used for getting robust inference in real visual scenes. They deal with such tasks as distinguishing different persons in a group, overcome problem with sufficient enclosures of human bodies by the foreground, detect various poses of people. In our work we use a new approach which enables to combine detection and classification tasks into one challenge using convolution neural networks. As a start point we choose YOLO CNN, whose authors propose a very efficient way of combining mentioned above tasks by learning a single neural network. This approach showed competitive results with state-of-the-art models such as FAST R-CNN, significantly overcoming them in speed, which allows us to apply it in real time video surveillance and other video monitoring systems. Despite all advantages it suffers from some known drawbacks, related to the fully-connected layers that obstruct applying the CNN to images with different resolution. Also it limits the ability to distinguish small close human figures in groups which is crucial for our tasks since we work with rather low quality images which often include dense small groups of people. In this work we gradually change network architecture to overcome mentioned above problems, train it on a complex pedestrian dataset and finally get the CNN detecting small pedestrians in real scenes.

Underwater Acoustic Source Localization and Sounds Classification in Distributed Measurement Networks

PurposeThe aim of this study was to analyse and review deep learning convolutional neural networks for detecting and diagnosing early-stage dental caries on periapical radiographs.Materials and MethodsIn order to conduct this review, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. Studies published from 2015 to 2021 under the keywords (deep convolutional neural network) AND (caries), (deep learning caries) AND (convolutional neural network) AND (caries) were systematically reviewed.ResultsWhen dental caries is improperly diagnosed, the lesion may eventually invade the enamel, dentin, and pulp tissue, leading to loss of tooth function. Rapid and precise detection and diagnosis are vital for implementing appropriate prevention and treatment of dental caries. Radiography and intraoral images are considered to play a vital role in detecting dental caries; nevertheless, studies have shown that 20% of suspicious areas are mistakenly diagnosed as dental caries using this technique; hence, diagnosis via radiography alone without an objective assessment is inaccurate. Identifying caries with a deep convolutional neural network-based detector enables the operator to distinguish changes in the location and morphological features of dental caries lesions. Deep learning algorithms have broader and more profound layers and are continually being developed, remarkably enhancing their precision in detecting and segmenting objects.ConclusionClinical applications of deep learning convolutional neural networks in the dental field have shown significant accuracy in detecting and diagnosing dental caries, and these models hold promise in supporting dental practitioners to improve patient outcomes.

Deep Learning Facilitates Automation of Wall Layer Quantification in Heart Transplant Coronary OCT

This study aims to compare the effectiveness of two deep learning models, namely Convolutional Neural Network (CNN) and combined CNN with Gated Recurrent Unit (CNN-GRU), in predicting Petrobras stock prices. Using historical stock price data from Yahoo Finance for the period 2017-2023, this study evaluates the performance of both models based on several evaluation metrics. The results showed that the CNN-GRU model outperformed the pure CNN model in all evaluation metrics, with an increase in RMSE value of 4.17% and an increase in R² value of 0.47% on the test data. The CNN-GRU model achieved 96.14% accuracy on the test data, while the CNN model achieved 96.04%. These findings indicate that the integration of CNN's feature extraction capabilities with GRU's temporal dependency modeling capabilities can improve stock price prediction accuracy. This research contributes to the computational finance literature by presenting an in-depth comparative analysis of the application of hybrid deep learning architectures in stock market prediction.

The world has recently seen unprecedented growth of social networks that have enabled internet users to share and communicate their views in various areas of life. Analysis of opinion is one type of text mining that aims to discover human views and emotions about any particular event, or topic. In the experiment of this research, we perform document level sentiment analysis in the form of positive and negative sentiments using both the traditional models and recent deep learning models. Three traditional methods (i.e. Naive Bayes (NB), Logistic Regression (LR), and Voting Ensemble) and three deep learning layout (i.e. Single Convolutional Neural Network (CNN), Single CNN + FastText and Ensemble(2-Layer CNN + Gated Recurrent Unit (GRU))) have been employed for two different datasets to evaluate the opinion classification. In our experiment, accuracy, f1-score, and RMSE have been considered as evaluation parameters and the optimistic results show that for airline review dataset, Single CNN + FastText comes out with the highest performance whereas for hotel review dataset, 2-L CNN + GRU comes out with the highest performance.

With the presentation of super-resolution convolutional neural network, deep learning approach was applied to image super-resolution reconstruction for the first time. By using convolutional neural network, the deep learning approaches can directly learn the mapping relationship between the low-resolution image and high-resolution image, and have achieved better reconstruction effects than the traditional image super-resolution reconstruction methods. Subsequently, a series of improved deep learning approaches have been proposed, and the reconstruction effects have been improved continuously. This paper systematically summa rizes the image super-resolution reconstruction approaches based on deep learning, analyzes the characteristics of different models, and compares the main deep learning models based on the experiments. Furthermore, based on deep learning model, the future research directions of the image super-resolution reconstruction methods based on deep learning models are reasonably predicted.

The purpose of this study was to develop and evaluate the performance of a convolutional neural network (CNN) that uses a novel A-line based classification approach to detect cancer in OCT images of breast specimens. Deep learning algorithms have been developed for OCT ophthalmology applications using pixel-based classification approaches. In this study, a novel deep learning approach was developed that classifies OCT A-lines of breast tissue. De-identified human breast tissues from mastectomy and breast reduction specimens were excised from patients at Columbia University Medical Center. A total of 82 specimens from 49 patients were imaged with OCT, including both normal tissues and non-neoplastic tissues. The proposed algorithm utilized a hybrid 2D/1D convolutional neural network (CNN) to map each single B-scan to a 1D label vector, which were derived from manual annotation. Each A-line was labelled as one of the following tissue types: ductal carcinoma in situ (DCIS), invasive ductal carcinoma (IDC), adipose, and stroma. Five-fold cross-validation Dice scores across tissue types were: 0.82-0.95 for IDC, 0.54-0.75 for DCIS, 0.67-0.91 for adipose, and 0.61-0.86 for stroma. In a second experiment, IDC and DCIS were combined as a single tissue class (malignancy) while stroma and adipose were combined as a second tissue class (non-malignancy). In this setup, the experiment yielded five-fold cross-validation Dice scores between 0.89-0.93, respectively. Future work includes acquiring more patient samples and to compare the algorithm to previous works, including both deep learning and traditional automatic image processing methods for classification of breast tissue in OCT images.

The use of machine learning (ML) has been increasing rapidly in the medical imaging field, including computer-aided diagnosis (CAD), radiomics, and medical image analysis. Recently, an ML area called deep learning emerged in the computer vision field and became very popular in many fields. It started from an event in late 2012, when a deep-learning approach based on a convolutional neural network (CNN) won an overwhelming victory in the best-known worldwide computer vision competition, ImageNet Classification. Since then, researchers in virtually all fields, including medical imaging, have started actively participating in the explosively growing field of deep learning. In this paper, the area of deep learning in medical imaging is overviewed, including (1) what was changed in machine learning before and after the introduction of deep learning, (2) what is the source of the power of deep learning, (3) two major deep-learning models: a massive-training artificial neural network (MTANN) and a convolutional neural network (CNN), (4) similarities and differences between the two models, and (5) their applications to medical imaging. This review shows that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is the learning of image data directly without object segmentation or feature extraction; thus, it is the source of the power of deep learning, although the depth of the model is an important attribute. The class of ML with image input (or image-based ML) including deep learning has a long history, but recently gained popularity due to the use of the new terminology, deep learning. There are two major models in this class of ML in medical imaging, MTANN and CNN, which have similarities as well as several differences. In our experience, MTANNs were substantially more efficient in their development, had a higher performance, and required a lesser number of training cases than did CNNs. "Deep learning", or ML with image input, in medical imaging is an explosively growing, promising field. It is expected that ML with image input will be the mainstream area in the field of medical imaging in the next few decades.

The purpose of this study is to evaluate and compare the performance of modern deep learning techniques for automatically recognizing and segmenting multiple organ regions on 3D CT images. CT image segmentation is one of the important task in medical image analysis and is still very challenging. Deep learning approaches have demonstrated the capability of scene recognition and semantic segmentation on nature images and have been used to address segmentation problems of medical images. Although several works showed promising results of CT image segmentation by using deep learning approaches, there is no comprehensive evaluation of segmentation performance of the deep learning on segmenting multiple organs on different portions of CT scans. In this paper, we evaluated and compared the segmentation performance of two different deep learning approaches that used 2D- and 3D deep convolutional neural networks (CNN) without- and with a pre-processing step. A conventional approach that presents the state-of-the-art performance of CT image segmentation without deep learning was also used for comparison. A dataset that includes 240 CT images scanned on different portions of human bodies was used for performance evaluation. The maximum number of 17 types of organ regions in each CT scan were segmented automatically and compared to the human annotations by using ratio of intersection over union (IU) as the criterion. The experimental results demonstrated the IUs of the segmentation results had a mean value of 79% and 67% by averaging 17 types of organs that segmented by a 3D- and 2D deep CNN, respectively. All the results of the deep learning approaches showed a better accuracy and robustness than the conventional segmentation method that used probabilistic atlas and graph-cut methods. The effectiveness and the usefulness of deep learning approaches were demonstrated for solving multiple organs segmentation problem on 3D CT images.