Deep learning in auditory attention decoding: A systematic review

The center of human activity and a key point of contact for social interaction between city people and the built environment is the street. With more street-level photos becoming available, urban landscape studies have more chances to examine and evaluate streetscapes closely and from various angles. This manuscript present the graph neural networks (GNN) optimized with gazelle optimization algorithm (GOA) for urban plants cape design based on large-scale street view (UPLS-GNN-GOA). Initially data is taken from ADE20K dataset. Afterward the data is fed to General Robust Subband Adaptive Filtering (GRSAF) based pre-processing process. The pre-processing output is given to Discrete Fractional Fourier Transforms (DFrFT) to extract the street greenery and openness of street canyons. After that, the extracted features are fed to Graph Neural Networks (GNN) to classify the public health, urban microclimate, human perception. The weight parameters of the GNN are optimized using Gazelle Optimization Algorithm (GOA).The UPLS-GNN-GOA method is implemented in Python, and its efficiency is determined using a series of performance evaluation metrics analysis, including accuracy, precision, recall, f1-score, and sensitivity. The proposed UPLS-GNN-GOA method shows the highest accuracy of 98%, precision of 98%, and F1-score of 97% while comparing other existing methods such as urban plantscape design based on convolutional neural network (UPLS-CNN), urban plantscape design based on deep convolutional neural network (UPLS-DCNN) and urban plantscape design based on deep learning and convolutional neural network (UPLS-DL-CNN) respectively.

The auditory attention decoding (AAD) approach was proposed to determine the identity of the attended talker in a multi-talker scenario by analyzing electroencephalography (EEG) data. Although the linear model-based method has been widely used in AAD, the linear assumption was considered oversimplified and the decoding accuracy remained lower for shorter decoding windows. Recently, nonlinear models based on deep neural networks (DNN) have been proposed to solve this problem. However, these models did not fully utilize both the spatial and temporal features of EEG, and the interpretability of DNN models was rarely investigated. In this paper, we proposed novel convolutional recurrent neural network (CRNN) based regression model and classification model, and compared them with both the linear model and the state-of-the-art DNN models. Results showed that, our proposed CRNN-based classification model outperformed others for shorter decoding windows (around 90% for 2 s and 5 s). Although worse than classification models, the decoding accuracy of the proposed CRNN-based regression model was about 5% greater than other regression models. The interpretability of DNN models was also investigated by visualizing layers' weight.

The exponential growth of digital payment systems and electronic commerce has positioned credit card fraud as one of the most pressing challenges facing modern financial institutions. While traditional rule-based detection systems have demonstrated effectiveness in controlled environments, they exhibit fundamental limitations in adaptability when confronted with the dynamic and adversarial nature of contemporary fraud schemes. This comprehensive review examines the evolution and current state of machine learning approaches for credit card fraud detection, spanning from classical supervised learning algorithms to cutting-edge deep learning architectures and graph-based methodologies. This paper addresses critical challenges inherent in fraud detection, including extreme class imbalance (fraudulent transactions typically represent less than 1% of total volumes), concept drift caused by evolving fraud strategies, asymmetric misclassification costs, and real-time processing constraints. We systematically analyze supervised learning methods such as Logistic Regression, Decision Trees, Random Forests, Support Vector Machines, and Gradient Boosting Machines (XGBoost), alongside deep learning architectures including Artificial Neural Networks, Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks, and hybrid CNN-LSTM models. A comparative analysis of existing studies from 2010 to 2025 reveals that ensemble methods and deep learning models consistently outperform traditional classifiers, with graph neural networks and attention-based architectures achieving state-of-the-art performance. Despite significant advancements, challenges related to model interpretability, adaptive learning in non-stationary environments, computational efficiency for real-time deployment, and data privacy compliance remain unresolved. This review identifies emerging research directions—including semi-supervised learning, adaptive feature selection, and explainable AI—as promising avenues for developing robust, scalable, and transparent fraud detection systems suitable for real-world financial environments.

Time series data are increasingly prevalent across domains such as finance, healthcare, manufacturing, and IoT, making accurate forecasting and anomaly detection critical for decision-making and system reliability. Traditional statistical methods (e.g., ARIMA, Holt-Winters) often fail to capture complex temporal dependencies and high-dimensional interactions inherent in modern time series. Recent advances in artificial intelligence particularly deep learning architectures such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), temporal convolutional networks (TCNs), graph neural networks (GNNs) and Transformers have demonstrated marked improvements in modeling both univariate and multivariate series, as well as in detecting anomalies that deviate from learned norms (Darban, Webb, Pan, Aggarwal, & Salehi, 2022; Chiranjeevi, Ramya, Balaji, Shashank, & Reddy, 2024) [1,2]. Moreover, ensemble techniques and hybrid signal-processing + deep-learning pipelines show enhanced sensitivity and adaptability in real-world anomaly detection scenarios (Iqbal, Amin, Alsubaei, & Alzahrani, 2024) [3]. In this work, we provide a unified survey and comparative analysis of AI-driven time series forecasting and anomaly detection methods, highlight key industrial application domains, evaluate performance trade-offs (e.g., accuracy vs. latency, supervised vs. unsupervised learning), and discuss emerging challenges including interpretability, data drift, real-time deployment on edge devices, and integration of causal reasoning. Our findings suggest that while AI approaches significantly outperform classical techniques in many settings, careful consideration of data characteristics, evaluation metrics and deployment environment remains essential for effective adoption.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Auditory attention decoding (AAD) is the process of identifying the attended speech in a multi-talker environment using brain signals, typically recorded through electroencephalography (EEG). Over the past decade, AAD has undergone continuous development, driven by its promising application in neuro-steered hearing devices. Most AAD algorithms are relying on the increase in neural entrainment to the envelope of attended speech, as compared to unattended speech, typically using a two-step approach. First, the algorithm predicts representations of the attended speech signal envelopes; second, it identifies the attended speech by finding the highest correlation between the predictions and the representations of the actual speech signals. In this study, we proposed a novel end-to-end neural network architecture, named AADNet, which combines these two stages into a direct approach to address the AAD problem. We compare the proposed network against traditional stimulus decoding-based approaches, including linear stimulus reconstruction, canonical correlation analysis, and an alternative non-linear stimulus reconstruction using three different datasets. AADNet shows a significant performance improvement for both subject-specific and subject-independent models. Notably, the average subject-independent classification accuracies for different analysis window lengths range from 56.3% (1 s) to 78.1% (20 s), 57.5% (1 s) to 89.4% (40 s), and 56.0% (1 s) to 82.6% (40 s) for three validated datasets, respectively, showing a significantly improved ability to generalize to data from unseen subjects. These results highlight the potential of deep learning models for advancing AAD, with promising implications for future hearing aids, assistive devices, and clinical assessments.

The study of graph neural networks has revealed that they can unleash new applications in a variety of disciplines using such a basic process that we cannot imagine in the context of other deep learning designs. Many limitations limit their expressiveness, and researchers are working to overcome them to fully exploit the power of graph data. There are a number of publications that explore graph neural networks (GNNs) restrictions and bottlenecks, but the common thread that runs through them all is that they can all be traced back to message passing, which is the key technique we use to train our graph models. We outline the general GNN design pipeline in this study as well as discuss solutions to the over-smoothing problem, categorize the solutions, and identify open challenges for further research. Abbreviations: CGNN: Continuous Graph Neural Networks; CNN: Convolution NeuralNetwork; DeGNN: Decomposition Graph Neural Network; DGN: Directional GraphNetworks; DGN: Differentiable Group Normalization; DL: Deep Learning; EGAI:Enhancing GNNs by a High-quality Aggregation of Beneficial Information; GAT: GraphAttention Network; GCN: Graph Convolutional Network; GDC: Graph Drop Connect; GDR: Group Distance Ratio; GNN: Graph Neural Network; GRAND: GraphRandom Neural Networks; IIG: Instance Information Gain; MAD: Man AverageDistance; PDE-GCN: Partial Differential Equations-GCN; PTDNet: ParameterizedTopological Denoising network; TDGNN: Tree Decomposition Graph NeuralNetwork;

The exponential growth in mobile data demand, fueled by the proliferation of smart devices, IoT applications, and multimedia services, has posed significant challenges to the performance andscalability of 5G networks. Traditional network traffic management techniques are increasingly insufficient to meet the dynamic and complex requirements of modern wireless communication systems. In response, deep learning (DL) has emerged as a powerful tool to enable intelligent, adaptive, and realtime optimization of mobile network traffic in 5G environments. This review provides a comprehensive overview of the integration of deep learning algorithms into various layers of the 5G architecture, including radio access networks (RAN), core networks, and edge computing platforms. Key DL models such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory (LSTM), deep reinforcement learning (DRL), and graph neural networks (GNNs) are analyzed in terms of their application to traffic prediction, congestion control, resource allocation, and quality of service (QoS) enhancement. The article also explores existing challenges such as data scarcity, model interpretability, latency constraints, and deployment complexity. Furthermore, it discusses emerging trends like federated learning, transfer learning, and edge intelligence as promising directions for future research. By synthesizing state-of-the-art contributions, this review highlights the transformative potential of DL in building efficient, resilient, and autonomous 5G networks.

This paper surveys the growing interest in utilizing deep learning (DL) as a powerful tool to address challenging problems in earthquake engineering. Despite decades of advancement in domain knowledge, issues such as uncertainty in earthquake occurrence, unpredictable seismic loads, nonlinear structural responses, and community engagement remain difficult to tackle using domain-specific methods. DL offers promising solutions by leveraging its data-driven capacity for nonlinear mapping, sequential data modeling, automatic feature extraction, dimensionality reduction, optimal decision-making, and so on. However, the literature lacks a comprehensive review that systematically covers a consistent scope intersecting DL and earthquake engineering. To bridge this gap, this paper first discusses methodological advances to elucidate various applicable DL techniques, such as multilayer perceptron, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network, autoencoder, transfer learning, reinforcement learning (RL), and graph neural network. A thorough research landscape is then disclosed by exploring various DL applications across different research topics, including vision-based seismic damage assessment and structural characterization, seismic demand and damage state prediction, seismic response history prediction, regional seismic risk assessment and community resilience, ground motion (GM) for engineering use, seismic response control, and the inverse problem of system/damage identification. Suitable DL techniques for each research topic are identified, emphasizing the preeminence of CNN for vision-based tasks, RNN for sequential data, RL for community resilience, and unsupervised learning for GM analysis. The paper also discusses opportunities and challenges for leveraging DL in earthquake engineering research and practice, highlighting the need for open-access multimodal big data and efforts to enhance model interpretability and incorporate physics information into DL. Finally, the paper advocates for DL applications to further advance the research frontier of uncertainty quantification in performance-based earthquake engineering.

Accurate calculation of drug-target affinity (DTA) is crucial for various applications in the pharmaceutical industry, including drug screening, design, and repurposing. However, traditional machine learning methods for calculating DTA often lack accuracy, posing a significant challenge in accurately predicting DTA. Fortunately, deep learning has emerged as a promising approach in computational biology, leading to the development of various deep learning-based methods for DTA prediction. To support researchers in developing novel and highly precision methods, we have provided a comprehensive review of recent advances in predicting DTA using deep learning. We firstly conducted a statistical analysis of commonly used public datasets, providing essential information and introducing the used fields of these datasets. We further explored the common representations of sequences and structures of drugs and targets. These analyses served as the foundation for constructing DTA prediction methods based on deep learning. Next, we focused on explaining how deep learning models, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Transformer, and Graph Neural Networks (GNNs), were effectively employed in specific DTA prediction methods. We highlighted the unique advantages and applications of these models in the context of DTA prediction. Finally, we conducted a performance analysis of multiple state-of-the-art methods for predicting DTA based on deep learning. The comprehensive review aimed to help researchers understand the shortcomings and advantages of existing methods, and further develop high-precision DTA prediction tool to promote the development of drug discovery.

Ensuring global food security requires accurate and robust solutions for crop health monitoring, weed detection, and large-scale land-cover classification. To this end, we propose AgroVisionNet, a hybrid deep learning framework that integrates Convolutional Neural Networks (CNNs) for local feature extraction, Vision Transformers (ViTs) for capturing long-range global dependencies, and Graph Neural Networks (GNNs) for modeling spatial relationships between image regions. The framework was evaluated on five diverse benchmark datasets—PlantVillage (leaf-level disease detection), Agriculture-Vision (field-scale anomaly segmentation), BigEarthNet (satellite-based land-cover classification), UAV Crop and Weed (weed segmentation), and EuroSAT (multi-class land-cover recognition). Across these datasets, AgroVisionNet consistently outperformed strong baselines including ResNet-50, EfficientNet-B0, ViT, and Mask R-CNN. For example, it achieved 97.8% accuracy and 95.6% IoU on PlantVillage, 94.5% accuracy on Agriculture-Vision, 92.3% accuracy on BigEarthNet, 91.5% accuracy on UAV Crop and Weed, and 96.4% accuracy on EuroSAT. These results demonstrate the framework’s robustness across tasks ranging from fine-grained disease detection to large-scale anomaly mapping. The proposed hybrid approach addresses persistent challenges in agricultural imaging, including class imbalance, image quality variability, and the need for multi-scale feature integration. By combining complementary architectural strengths, AgroVisionNet establishes a new benchmark for deep learning applications in precision agriculture.

Edge AI systems are increasingly being adopted in a wide range of application domains. These systems typically deploy Convolutional Neural Network (CNN) models on edge devices to perform inference, while relying on the cloud for model training. This is due to the high computational and memory demands of conventional model training, which exceeds the capabilities of resource-constrained edge devices running on tight power budgets. The dependency on the cloud for training is not suitable in many applications, where new objects or environmental conditions different from the ones present during training, are frequently encountered. In such applications, continual learning of new knowledge on the edge device becomes a necessity to avoid performance bottlenecks due to round-trip communication delays, network connectivity, and the available bandwidth. In this thesis, we propose Field-Programmable Gate Array (FPGA) based accelerator architecture and optimization strategies for a new paradigm of machine learning algorithms that is capable of continual learning. The proposed methods will enable edge FPGA systems to perform on-device deep continual learning for object classification. Specifically, the proposed methods aim to achieve real-time learning on-device, while providing for a high degree of scalability to learn a large number of classes. We first propose a FPGA accelerator for a Self-Organizing Neural Network (SONN), that can perform class-incremental continual learning in a streaming manner when combined with a CNN. The SONN model performs unsupervised learning from embedding features extracted from the CNN model by dynamically growing neurons and connections. We introduce design optimization strategies and runtime scheduling techniques to optimize resource usage, latency, and energy consumption. Experimental results based on Core50 dataset for continuous object recognition from video sequences, demonstrated that the proposed FPGA architecture outperforms CPU and GPU based counterparts in terms of latency and power. However, the SONN model grows proportionally to the classes learnt, which limits its scalability to learn a large number of classes efficiently. Next, we propose a FPGA accelerator for a Streaming Linear Discriminant Analysis (SLDA) model to overcome the scalability limitations of SONN. Similar to the SONN, SLDA performs continual learning from embedding features extracted from a CNN in a streaming manner. SLDA is highly scalable for learning a large number of classes as the network does not grow dynamically to accommodate new knowledge. We propose several design and runtime optimizations to minimize resource usage, latency, and energy consumption. Additionally, we introduce a new variant of SLDA and discuss the accuracy-efficiency trade-offs using popular datasets for continual learning, CoRE50 and CUB200. The results demonstrate that the proposed SLDA accelerator outperforms CPU and GPU counterparts, in terms of latency and energy efficiency. Finally, we demonstrate a full on-chip deep continual learning pipeline on FPGA, by integrating the proposed SLDA accelerator with Xilinx DPU, a programmable CNN accelerator IP. The design is implemented on a Xilinx Zynq Ultrascale+ MPSoC. In order to overcome the large performance bottleneck due to the communication overhead between the ARM processing system (PS) and programmable logic (PL), we implemented a Linux device driver that facilitates efficient memory mapping between PS-PL. The experimental results based on CoRE50 dataset show that the proposed pipeline is capable of performing continual learning at nearly the same latency as the inference pipeline, with only a marginal increase in energy consumption. Our results clearly demonstrate the viability of deploying real-time deep continual learning on edge AI systems that are equipped with FPGA accelerators.

At present, palmprint recognition based on deep learning has been more and more widely used in identity recognition due to its many advantages. However, these algorithms often require a large amount of labelled data for training. In fact, it is difficult and expensive to get enough data that meets the requirements. In this Letter, based on a small amount of labelled images, the authors proposed a method for few‐shot palmprint recognition using Graph Neural Networks (GNNs). The palmprint features extracted by the convolutional neural network are processed into nodes in the GNN. The edges in the GNN are used to represent similarities between image nodes. The parameters in the network are continuously optimised, and finally, the category to which each image belongs is obtained. Further, they adopted a mobile phone to create a palmprint database in an unconstrained way. Adequate experiments were performed on the benchmark database and the authors’ self‐built database. The experimental results show that their proposed GNN‐based few‐shot palmprint recognition can obtain state‐of‐the‐art performance, where the accuracy is over 99.90%.

A multi-agent artificial intelligence (AI) model is developed to automate the discovery of new metallic alloys, integrating multimodal data and external knowledge, including insights from physics via atomistic simulations. The system consists of (a) large language models (LLMs) for tasks such as reasoning and planning, (b) AI agents with distinct roles collaborating dynamically, and (c) a newly developed graph neural network (GNN) model for rapid retrieval of physical properties. We chose the ternary NbMoTa body-centered-cubic alloy as our model system and developed the GNN to predict two fundamental materials properties: the Peierls barrier and the solute/screw dislocation interaction energy. Our GNN model efficiently predicts these properties, reducing reliance on costly brute-force calculations and alleviating the computational demands on the multi-agent system. By combining the predictive capabilities of GNNs with the collaborative intelligence of LLM-driven reasoning agents, the system autonomously explores vast alloy design spaces, identifies trends in atomic-scale properties, and predicts macroscale mechanical strength, as demonstrated by several computational experiments. This synergistic approach accelerates the discovery of advanced alloys and holds promise for broader applications in other complex systems, marking a step forward in automated materials discovery and design. Impact statement Traditional deep learning models, such as graph neural networks and convolutional neural networks, operate within the confines of their training data sets, making single-step inferences for regression or classification. Our work introduces a multi-agent strategy that transcends these limitations by integrating deep learning with reasoning and decision-making capabilities. This intelligent system actively interprets results, determines subsequent actions, and iteratively refines predictions, accelerating the materials design process. We demonstrate its effectiveness in exploring the vast compositional space of a ternary alloy, where the model dynamically solicits data, analyzes trends, generates visualizations, and derives insights into materials behavior. By enabling accurate predictions of key alloy characteristics, our approach advances the discovery of novel metallic systems and underscores the critical role of solid-solution alloying. More broadly, it represents a major step toward integrating artificial intelligence with scientific reasoning, moving closer to artificial general intelligence in engineering. This paradigm shift has profound implications for materials science, enabling more efficient, autonomous, and intelligent exploration of complex materials spaces. Graphical Abstract

Remaining Useful Life (RUL) prediction is a critical technology in prognostics and health management. Rotating machinery plays an indispensable role in industrial processes, and accurate RUL prediction for such equipment is essential to ensure production continuity, optimize maintenance strategies and reduce operational costs. In recent years, deep learning has emerged as a powerful tool for the analysis of multi-source monitoring data derived from rotating machinery, owing to its exceptional proficiency in feature extraction and non-linear modelling. This approach can effectively excavate the deeper-level information within the data, thereby improving prediction accuracy. Consequently, deep learning demonstrates significant application value and broad development prospects in RUL prediction for rotating machinery. This paper provides a comprehensive review of the application of deep learning techniques in RUL prediction for rotating machinery. Firstly, it introduces a unified framework for RUL prediction based on deep learning methodologies. The article then elaborates on the theoretical foundations of several key learning methods, including autoencoders, convolutional neural networks, long short-term memory, gated recurrent units, as well as emerging models such as transformers, temporal convolutional networks and graph neural networks. Subsequently, the paper summarizes the applications of these deep learning methods in RUL prediction for rotating machinery, evaluating the specific problems each method addresses based on their distinct characteristics. Finally, it discusses the significant challenges that deep learning faces within this domain and outlines potential directions for future research.