A deep learning analysis on question classification task using Word2vec representations

Deep learning systems for automatic diagnosis of infant cry signals

This study compared popular Deep Learning (DL) architectures to classify machining surface roughness using sound and force data. The DL architectures considered in this study include Multi-Layer Perceptron (MLP), Convolution Neural Network (CNN), Long Short-Term Memory (LSTM), and transformer. The classification was performed on the sound and force data generated during machining aluminum sheets for different levels of spindle speed, feed rate, depth of cut, and end-mill diameter, and it was trained on 30 s machining data (10–40 s) of the machining experiments. Since a raw audio waveform is seldom used in DL models, Mel-Spectrogram and Mel Frequency Cepstral Coefficients (MFCCs) audio feature extraction techniques were used in the DL models. The results of DL models were compared for the training–validation accuracy, training epochs, and training parameters of each model. Although the roughness classification by all the DL models was satisfactory (except for CNN with Mel-Spectrogram), the transformer-based modes had the highest training (>96%) and validation accuracies (≈90%). The CNN model with Mel-Spectrogram exhibited the worst training and inference accuracy, which is influenced by limited training data. Confusion matrices were plotted to observe the classification accuracy visually. The confusion matrices showed that the transformer model trained on Mel-Spectrogram and the transformer model trained on MFCCs correctly predicted 366 (or 91.5%) and 371 (or 92.7%) out of 400 test samples. This study also highlights the suitability and superiority of the transformer model for time series sound and force data and over other DL models.

In this paper, we develop a method based on a deep learning method in financial market prediction, which includes BRICS economies as the test cases. Financial markets are rife with volatility that is affected by a "bed of complexity," coddled by local and distal factors. To leverage these vast datasets both deep learning models such as Convolutional Neural Networks (CNNs), Long Short Term Memory (LSTM) networks as well as hybrid architectures are used in this study. The paper evaluates the predictive accuracy of the models, and by so doing, identifies their strengths in predicting temporal dependencies and intricate market patterns. In particular, deep learning techniques are applied to case studies of individual countries in the BRICS to highlight the application of deep learning to disparate country specific problems, such as liquidity crises and market shocks. These findings show that classical statistical methods are outperformed by deep learning systems in a precise and reliable financial forecasting. This research highlights the ability of AI driven systems to change financial decision making processes, improving investor confidence and improving economic stability in BRICS nations. This study also readers the value of deep learning in financial market analysis, especially in economies in the developing countries. Application of techniques and architectures e.g. Convolutional Neural Networks (CNNs) that excel at identifying spatial patterns, and Long Short-Term Memory (LSTM) networks renowned for their prowess on sequential and time series data, for real world market prediction are explained. In addition, the research discusses hybrid architectures which extend knowledge, fusing strengths of both architectures to improve prediction accuracy and how deep learning develops to solve particular financial challenges. Through reading these notes readers get exposed to data preprocessing techniques such as normalization and feature selection which are important for boosting deep learning performance. The paper also includes an introduction to the evaluation of models using MSE and R-squared values for validating them in terms of reliable outputs. This research combines deep learning theory and practical case study to offer a useful educational resource for students, researchers, and practitioners who want to apply AI in financial forecasting in complex and dynamic global markets.

Classic image scaling (e.g. bicubic) can be seen as one convolutional layer and a single upscaling filter. Its implementation is ubiquitous in all display devices and image processing software. In the last decade deep learning systems have been introduced for the task of image super-resolution (SR), using several convolutional layers and numerous filters. These methods have taken over the benchmarks of image quality for upscaling tasks. Would it be possible to replace classic upscalers with deep learning architectures on edge devices such as display panels, tablets, laptop computers, etc.? On one hand, the current trend in Edge–AI chips shows a promising future in this direction, with rapid development of hardware that can run deep–learning tasks efficiently. On the other hand, in image SR only few architectures have pushed the limit to extreme small sizes that can actually run on edge devices at real-time. We explore possible solutions to this problem with the aim to fill the gap between classic upscalers and small deep learning configurations. As a transition from classic to deep–learning upscaling we propose edge–SR (eSR), a set of one–layer architectures that use interpretable mechanisms to upscale images. Certainly, a one–layer architecture cannot reach the quality of deep learning systems. Nevertheless, we find that for high speed requirements, eSR becomes better at trading–off image quality and runtime performance. Filling the gap between classic and deeplearning architectures for image upscaling is critical for massive adoption of this technology. It is equally important to have an interpretable system that can reveal the inner strategies to solve this problem and guide us to future improvements and better understanding of larger networks.

This study proposes a compact deep learning (DL) architecture and a highly parallelized computing hardware platform to reconstruct the blood flow index (BFi) in diffuse correlation spectroscopy (DCS). We leveraged a rigorous analytical model to generate autocorrelation functions (ACFs) to train the DL network. We assessed the accuracy of the proposed DL using simulated and milk phantom data. Compared to convolutional neural networks (CNN), our lightweight DL architecture achieves 66.7% and 18.5% improvement in MSE for BFi and the coherence factor β, using synthetic data evaluation. The accuracy of rBFi over different algorithms was also investigated. We further simplified the DL computing primitives using subtraction for feature extraction, considering further hardware implementation. We extensively explored computing parallelism and fixed-point quantization within the DL architecture. With the DL model's compact size, we employed unrolling and pipelining optimizations for computation-intensive for-loops in the DL model while storing all learned parameters in on-chip BRAMs. We also achieved pixel-wise parallelism, enabling simultaneous, real-time processing of 10 and 15 autocorrelation functions on Zynq-7000 and Zynq-UltraScale+ field programmable gate array (FPGA), respectively. Unlike existing FPGA accelerators that produce BFi and the β from autocorrelation functions on standalone hardware, our approach is an encapsulated, end-to-end on-chip conversion process from intensity photon data to the temporal intensity ACF and subsequently reconstructing BFi and β. This hardware platform achieves an on-chip solution to replace post-processing and miniaturize modern DCS systems that use single-photon cameras. We also comprehensively compared the computational efficiency of our FPGA accelerator to CPU and GPU solutions.

Steering angle prediction plays a crucial role in the control of Autonomous Vehicles (AVs) and has garnered the interest of researchers, manufacturers, and insurance companies within the automotive industry. Various Deep Learning (DL) architectures have been utilized to forecast the steering angle of AVs across different scenarios. An examination focusing on steering angle prediction through deep learning algorithms can assist seasoned researchers in pinpointing areas that necessitate further advancement. Additionally, newcomers in the field can utilize this study as a foundation. This paper offers an extensive analysis of the recent progress achieved in DL architectures concerning the steering angle prediction of AVs. A comprehensive taxonomy outlining the application of DL in steering angle prediction of AVs has been developed. The survey provides a succinct synthesis, summary, and analysis of research findings. It is evident that Convolutional Neural Network (CNN) is the preferred choice among researchers for predicting the steering angle of autonomous vehicles, compared to other DL architectures. Furthermore, existing research challenges have been identified. The primary obstacle encountered in DL-based steering angle prediction of AVs is the scarcity of real-world datasets, leading researchers to heavily rely on data generated from simulated environments. Lastly, potential alternative approaches to address the identified research challenges have been suggested, indicating promising avenues for future research endeavours.

Deep learning based classification of breast tumors with shear-wave elastography

The aims of this work are (1) to explore deep learning (DL) architectures, spectroscopic input types, and learning designs toward optimal quantification in MR spectroscopy of simulated pathological spectra; and (2) to demonstrate accuracy and precision of DL predictions in view of inherent bias toward the training distribution. Simulated 1D spectra and 2D spectrograms that mimic an extensive range of pathological in vivo conditions are used to train and test 24 different DL architectures. Active learning through altered training and testing data distributions is probed to optimize quantification performance. Ensembles of networks are explored to improve DL robustness and reduce the variance of estimates. A set of scores compares performances of DL predictions and traditional model fitting (MF). Ensembles of heterogeneous networks that combine 1D frequency-domain and 2D time-frequency domain spectrograms as input perform best. Dataset augmentation with active learning can improve performance, but gains are limited. MF is more accurate, although DL appears to be more precise at low SNR. However, this overall improved precision originates from a strong bias for cases with high uncertainty toward the dataset the network has been trained with, tending toward its average value. MF mostly performs better compared to the faster DL approach. Potential intrinsic biases on training sets are dangerous in a clinical context that requires the algorithm to be unbiased to outliers (i.e., pathological data). Active learning and ensemble of networks are good strategies to improve prediction performances. However, data quality (sufficient SNR) has proven as a bottleneck for adequate unbiased performance-like in the case of MF.

The product review dataset is a rapidly growing and interesting source of data to explore. The increase in the number of internet users and customer shopping habits through online stores has a significant impact on the growth of product review data, especially for online stores in Indonesia, such as Tokopedia. The sample data used amounted to 1079. This research aims to evaluate the performance of three types of pre-trained word embeddings, namely FastText, GloVe, and Word2Vec, in the Long Short-Term Memory (LSTM) model for sentiment classification of product reviews on Tokopedia. An automated sentiment classification system is needed to process many product reviews, making it easier for sellers to know what consumers think of their products. This research contributes by evaluating the impact of various pre-trained word embeddings on the performance of LSTM models in sentiment classification tasks. In addition, this research also aims to measure the effectiveness of LSTM models combined with multiple pre-trained word embeddings. By implementing a deep learning architecture, computers can learn and recognize contextual data stored in review sentences. The research was conducted in three stages: model selection, layer setup, and hyperparameter optimization, to feature in-depth testing of the deep learning architecture used and the appropriate combination of layers and parameters to obtain high sentiment classification performance. The experimental results show that FastText with LSTM provides the best performance with 85.08% accuracy, Word2Vec with 84.62% accuracy, and GloVe with 83.04% accuracy. The main contribution of this research is to present an in-depth test of the product review dataset and provide a deep learning architecture along with a combination of layers and parameters that has the best performance in recognizing sentiment on the product review dataset. This architecture achieves higher performance than the BERT method with CNN and BiLSTM layers.

Building energy management is crucial in reducing energy consumption and maintaining occupant comfort, especially in heating systems. However, achieving optimal space heating efficiency while maintaining consistent comfort presents significant challenges. Traditional methods often fail to balance energy consumption with thermal comfort, especially across multiple zones in buildings with varying operational demands. This study investigates the role of deep learning models in optimizing space heating while maintaining thermal comfort across multiple building zones. It aims to enhance heating efficiency by developing predictive models for building temperature and heating consumption, evaluating the effectiveness of different deep learning architectures, and analyzing the impact of model-driven heating optimization on energy savings and occupant comfort. To address this challenge, this study employs Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Transformer models to forecast area temperatures and predict space heating consumption. The proposed methodology leverages historical building temperature data, weather station measurements such as atmospheric pressure, wind speed, wind direction, relative humidity, and solar radiation, along with other weather parameters, to develop accurate and reliable predictions. A two-stage deep learning process is utilized: first, temperature predictions are generated for different building zones, and second, these predictions are used to estimate global heating consumption. This study also employs grid search and cross-validation to optimize the model configurations and custom loss functions to ensure energy efficiency and occupant comfort. Results demonstrate that the Long Short-Term Memory and Transformer models outperform the Gated Recurrent Unit regarding heating reduction, with a 20.95% and 20.69% decrease, respectively, compared to actual consumption. This study contributes significantly to energy management by providing a deep learning-driven framework that enhances energy efficiency while maintaining thermal comfort across different building areas, thereby supporting sustainable and intelligent building operations.

Albeit deep learning has chronicled roots and has been applied to computer vision task since 2000 but a decade ago, neither the expression “Deep learning” nor the methodology was well known. This dormant field regains consciousness when a highly influential paper “Image Net Classification with Deep Convolutional Neural Networks by Krizhevsky, Sutskever and Hinton’in 2012” was published. Now, availability of abundance of data, computational power, and improved algorithms has contributed altogether and brought this technology to forefront in the field of machine learning. In this paper, we focus on growth of various convolution neural network architectures (deep learning architectures), from their predecessors up to recent state-of-the-art deep learning systems. The paper has three sections: (1) Introduction about neural networks along with necessary back ground information. (2) Hierarchy of classical and modern architectures; In this section, the existing methods are explained and their contribution and significance in field of machine learning are highlighted. At last, we point out a set of promising future works and draw our own conclusions.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

The prediction of stock price movement direction is significant in financial circles and academic. Stock price contains complex, incomplete, and fuzzy information which makes it an extremely difficult task to predict its development trend. Predicting and analysing financial data is a nonlinear, time-dependent problem. With rapid development in machine learning and deep learning, this task can be performed more effectively by a purposely designed network. This paper aims to improve prediction accuracy and minimizing forecasting error loss through deep learning architecture by using Generative Adversarial Networks. It was proposed a generic model consisting of Phase-space Reconstruction (PSR) method for reconstructing price series and Generative Adversarial Network (GAN) which is a combination of two neural networks which are Long Short-Term Memory (LSTM) as Generative model and Convolutional Neural Network (CNN) as Discriminative model for adversarial training to forecast the stock market. LSTM will generate new instances based on historical basic indicators information and then CNN will estimate whether the data is predicted by LSTM or is real. It was found that the Generative Adversarial Network (GAN) has performed well on the enhanced root mean square error to LSTM, as it was 4.35% more accurate in predicting the direction and reduced processing time and RMSE by 78 secs and 0.029, respectively. This study provides a better result in the accuracy of the stock index. It seems that the proposed system concentrates on minimizing the root mean square error and processing time and improving the direction prediction accuracy, and provides a better result in the accuracy of the stock index.

In this study, we investigate various deep learning models based on convolutional neural networks (CNNs) and Long Short Term Memory (LSTM) recurrent neural networks for sentiment analysis of Arabic microblogs. Unlike English, the Arabic language has several specifics which complicate the process of feature extraction by traditional methods. We adopted a neural language model created at Google, known as word2vec, for vectorizing text. We then designed and evaluated several deep learning architectures using CNN and LSTM. The experiments were run on two publicly available Arabic tweets datasets. Promising results have been attained when combining LSTMs and compared favorably with most related work.

Detection of smoke from videos captured by surveillance cameras in outdoor environments is one of the useful outcome of Internet of Things (IoT) applications. The potential benefit increases when deep learning (DL) architectures are involved. However, an inherent difficulty is to detect smoke while natural events like fog exists. The effectiveness of color spaces in detection performance has not yet fully evaluated in those architectures. Moreover, the energy and memory requirements of DL architectures may not be applicable for handling IoT implementation demands. Therefore, in this work, a DL architecture with a suitable color space model, applicable for IoT implementations is proposed to detect smoke from videos in foggy environment. By collecting several videos including smoke samples, the performance comparison of popular and the state-of-the-art DL architectures denoted the outperforming result according to both accuracy and memory usage.