Arabic Legal Text Classification Using Pre-Trained Transformers and Deep Learning

AI-based emotion recognition has emerged as a critical component of affect-aware educational technologies, particularly in online, large-scale, and technology-mediated learning environments where direct observation of learners’ emotional states is limited. By leveraging facial expressions, speech, physiological signals, and interaction data, emotion-aware systems aim to support adaptive instruction, learner engagement, and instructional decision-making. Despite rapid technical progress, the integration of emotion recognition into authentic educational practice remains constrained by dataset limitations, cultural bias, weak pedagogical alignment, and unresolved ethical concerns. This article presents a critical and theory-informed review of recent advances in AI-based emotion recognition in education, with particular emphasis on deep learning architectures and multimodal fusion approaches, while highlighting persistent challenges and future research directions.Purpose:This study aims to critically examine the current state of AI-based emotion recognition in educational contexts by analyzing dominant modeling paradigms, sensing modalities, application domains, and ethical implications, in order to identify key gaps between technical development and pedagogical practice.Methods/Study design/approach:The study adopts a narrative literature review approach grounded in educational and affective computing theories. Representative and influential studies were analyzed across facial, speech, physiological, and multimodal emotion recognition systems, with emphasis on deep learning, attention-based, and transformer architectures. The review synthesizes findings related to system performance, educational applications, dataset characteristics, and ethical considerations.Result/Findings:The findings indicate that multimodal emotion recognition systems generally outperform unimodal approaches in detecting learning-relevant affective states such as engagement, frustration, and confusion. Advances in deep learning, particularly attention-based and transformer models, improve temporal modeling and robustness. However, most systems rely on non-educational datasets, demonstrate limited cultural generalizability, and prioritize recognition accuracy over pedagogical interpretability and long-term educational impact. Ethical challenges related to privacy, consent, bias, and emotional surveillance remain insufficiently addressed.Novelty/Originality/Value:This study provides an integrative and pedagogically grounded synthesis that bridges affective computing research with educational theory and governance perspectives. By reframing AI-based emotion recognition as a decision-support mechanism rather than an automated control system, the article offers a conceptual foundation for responsible, culturally inclusive, and ethically aligned deployment of emotion-aware AI in education, serving as a reference for future interdisciplinary research and policy development.

Since radiology reports needed for clinical practice and research are written and stored in free-text narrations, extraction of relative information for further analysis is difficult. In these circumstances, natural language processing (NLP) techniques can facilitate automatic information extraction and transformation of free-text formats to structured data. In recent years, deep learning (DL)-based models have been adapted for NLP experiments with promising results. Despite the significant potential of DL models based on artificial neural networks (ANN) and convolutional neural networks (CNN), the models face some limitations to implement in clinical practice. Transformers, another new DL architecture, have been increasingly applied to improve the process. Therefore, in this study, we propose a transformer-based fine-grained named entity recognition (NER) architecture for clinical information extraction. We collected 88 abdominopelvic sonography reports in free-text formats and annotated them based on our developed information schema. The text-to-text transfer transformer model (T5) and Scifive, a pre-trained domain-specific adaptation of the T5 model, were applied for fine-tuning to extract entities and relations and transform the input into a structured format. Our transformer-based model in this study outperformed previously applied approaches such as ANN and CNN models based on ROUGE-1, ROUGE-2, ROUGE-L, and BLEU scores of 0.816, 0.668, 0.528, and 0.743, respectively, while providing an interpretable structured report.

This study compared popular Deep Learning (DL) architectures to classify machining surface roughness using sound and force data. The DL architectures considered in this study include Multi-Layer Perceptron (MLP), Convolution Neural Network (CNN), Long Short-Term Memory (LSTM), and transformer. The classification was performed on the sound and force data generated during machining aluminum sheets for different levels of spindle speed, feed rate, depth of cut, and end-mill diameter, and it was trained on 30 s machining data (10–40 s) of the machining experiments. Since a raw audio waveform is seldom used in DL models, Mel-Spectrogram and Mel Frequency Cepstral Coefficients (MFCCs) audio feature extraction techniques were used in the DL models. The results of DL models were compared for the training–validation accuracy, training epochs, and training parameters of each model. Although the roughness classification by all the DL models was satisfactory (except for CNN with Mel-Spectrogram), the transformer-based modes had the highest training (>96%) and validation accuracies (≈90%). The CNN model with Mel-Spectrogram exhibited the worst training and inference accuracy, which is influenced by limited training data. Confusion matrices were plotted to observe the classification accuracy visually. The confusion matrices showed that the transformer model trained on Mel-Spectrogram and the transformer model trained on MFCCs correctly predicted 366 (or 91.5%) and 371 (or 92.7%) out of 400 test samples. This study also highlights the suitability and superiority of the transformer model for time series sound and force data and over other DL models.

This study proposes a compact deep learning (DL) architecture and a highly parallelized computing hardware platform to reconstruct the blood flow index (BFi) in diffuse correlation spectroscopy (DCS). We leveraged a rigorous analytical model to generate autocorrelation functions (ACFs) to train the DL network. We assessed the accuracy of the proposed DL using simulated and milk phantom data. Compared to convolutional neural networks (CNN), our lightweight DL architecture achieves 66.7% and 18.5% improvement in MSE for BFi and the coherence factor β, using synthetic data evaluation. The accuracy of rBFi over different algorithms was also investigated. We further simplified the DL computing primitives using subtraction for feature extraction, considering further hardware implementation. We extensively explored computing parallelism and fixed-point quantization within the DL architecture. With the DL model's compact size, we employed unrolling and pipelining optimizations for computation-intensive for-loops in the DL model while storing all learned parameters in on-chip BRAMs. We also achieved pixel-wise parallelism, enabling simultaneous, real-time processing of 10 and 15 autocorrelation functions on Zynq-7000 and Zynq-UltraScale+ field programmable gate array (FPGA), respectively. Unlike existing FPGA accelerators that produce BFi and the β from autocorrelation functions on standalone hardware, our approach is an encapsulated, end-to-end on-chip conversion process from intensity photon data to the temporal intensity ACF and subsequently reconstructing BFi and β. This hardware platform achieves an on-chip solution to replace post-processing and miniaturize modern DCS systems that use single-photon cameras. We also comprehensively compared the computational efficiency of our FPGA accelerator to CPU and GPU solutions.

Building energy management is crucial in reducing energy consumption and maintaining occupant comfort, especially in heating systems. However, achieving optimal space heating efficiency while maintaining consistent comfort presents significant challenges. Traditional methods often fail to balance energy consumption with thermal comfort, especially across multiple zones in buildings with varying operational demands. This study investigates the role of deep learning models in optimizing space heating while maintaining thermal comfort across multiple building zones. It aims to enhance heating efficiency by developing predictive models for building temperature and heating consumption, evaluating the effectiveness of different deep learning architectures, and analyzing the impact of model-driven heating optimization on energy savings and occupant comfort. To address this challenge, this study employs Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Transformer models to forecast area temperatures and predict space heating consumption. The proposed methodology leverages historical building temperature data, weather station measurements such as atmospheric pressure, wind speed, wind direction, relative humidity, and solar radiation, along with other weather parameters, to develop accurate and reliable predictions. A two-stage deep learning process is utilized: first, temperature predictions are generated for different building zones, and second, these predictions are used to estimate global heating consumption. This study also employs grid search and cross-validation to optimize the model configurations and custom loss functions to ensure energy efficiency and occupant comfort. Results demonstrate that the Long Short-Term Memory and Transformer models outperform the Gated Recurrent Unit regarding heating reduction, with a 20.95% and 20.69% decrease, respectively, compared to actual consumption. This study contributes significantly to energy management by providing a deep learning-driven framework that enhances energy efficiency while maintaining thermal comfort across different building areas, thereby supporting sustainable and intelligent building operations.

Steering angle prediction plays a crucial role in the control of Autonomous Vehicles (AVs) and has garnered the interest of researchers, manufacturers, and insurance companies within the automotive industry. Various Deep Learning (DL) architectures have been utilized to forecast the steering angle of AVs across different scenarios. An examination focusing on steering angle prediction through deep learning algorithms can assist seasoned researchers in pinpointing areas that necessitate further advancement. Additionally, newcomers in the field can utilize this study as a foundation. This paper offers an extensive analysis of the recent progress achieved in DL architectures concerning the steering angle prediction of AVs. A comprehensive taxonomy outlining the application of DL in steering angle prediction of AVs has been developed. The survey provides a succinct synthesis, summary, and analysis of research findings. It is evident that Convolutional Neural Network (CNN) is the preferred choice among researchers for predicting the steering angle of autonomous vehicles, compared to other DL architectures. Furthermore, existing research challenges have been identified. The primary obstacle encountered in DL-based steering angle prediction of AVs is the scarcity of real-world datasets, leading researchers to heavily rely on data generated from simulated environments. Lastly, potential alternative approaches to address the identified research challenges have been suggested, indicating promising avenues for future research endeavours.

Deep learning based classification of breast tumors with shear-wave elastography

This study proposes a modified and adaptive Balanced Scorecard (BSC) framework tailored to the strategic context of art museums as its primary contribution. The adapted BSC categorizes museum strategies into four dimensions—Social Values and Cultural Impact (SVCI), Visitor Experience and Tourism Development (VED), Exhibition and Operational Efficiency (EOE), and Art/Cultural Education and Sustainable Growth (ACES)—providing a systematic lens for interpreting and quantifying qualitative managerial priorities. These strategic perspectives were extracted from unstructured textual data sourced from museum websites and visitor-generated online content through natural language processing and text mining techniques. Building on this strategic classification, the study develops deep learning–based predictive models that integrate both structured numerical data and text-derived qualitative variables to estimate visitor numbers. Eight deep learning architectures, including Transformer, Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), and Generative Adversarial Network (GAN), were evaluated. The results indicate that VED exerts the most substantial influence on visitor attendance, particularly through interactive exhibitions, personalized experiences, and tourism collaborations, while other strategic dimensions provide marginal improvements in predictive accuracy. The Transformer model achieved the best predictive performance, reflecting its superior ability to capture complex relationships between structured and unstructured data. By combining an adapted BSC framework with advanced predictive modeling, this study offers both a novel theoretical approach for categorizing museum strategies and a practical tool for enhancing decision-making in visitor engagement, exhibition planning, and resource allocation.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

The aims of this work are (1) to explore deep learning (DL) architectures, spectroscopic input types, and learning designs toward optimal quantification in MR spectroscopy of simulated pathological spectra; and (2) to demonstrate accuracy and precision of DL predictions in view of inherent bias toward the training distribution. Simulated 1D spectra and 2D spectrograms that mimic an extensive range of pathological in vivo conditions are used to train and test 24 different DL architectures. Active learning through altered training and testing data distributions is probed to optimize quantification performance. Ensembles of networks are explored to improve DL robustness and reduce the variance of estimates. A set of scores compares performances of DL predictions and traditional model fitting (MF). Ensembles of heterogeneous networks that combine 1D frequency-domain and 2D time-frequency domain spectrograms as input perform best. Dataset augmentation with active learning can improve performance, but gains are limited. MF is more accurate, although DL appears to be more precise at low SNR. However, this overall improved precision originates from a strong bias for cases with high uncertainty toward the dataset the network has been trained with, tending toward its average value. MF mostly performs better compared to the faster DL approach. Potential intrinsic biases on training sets are dangerous in a clinical context that requires the algorithm to be unbiased to outliers (i.e., pathological data). Active learning and ensemble of networks are good strategies to improve prediction performances. However, data quality (sufficient SNR) has proven as a bottleneck for adequate unbiased performance-like in the case of MF.

Detection of smoke from videos captured by surveillance cameras in outdoor environments is one of the useful outcome of Internet of Things (IoT) applications. The potential benefit increases when deep learning (DL) architectures are involved. However, an inherent difficulty is to detect smoke while natural events like fog exists. The effectiveness of color spaces in detection performance has not yet fully evaluated in those architectures. Moreover, the energy and memory requirements of DL architectures may not be applicable for handling IoT implementation demands. Therefore, in this work, a DL architecture with a suitable color space model, applicable for IoT implementations is proposed to detect smoke from videos in foggy environment. By collecting several videos including smoke samples, the performance comparison of popular and the state-of-the-art DL architectures denoted the outperforming result according to both accuracy and memory usage.

Multi-label text classification is a natural extension of text classification in which each document can be assigned with a possible widespread set of labels. Natural Language Processing (NLP) helps to understand and manipulate text in natural language by using the computer. Arabic Text Classification is challenging recently because the Arabic language is under-resourced although it has many users. The aim of this paper is to build a model to classify Arabic news and help users get and display the most relevant news to their interests. In this paper, we demonstrate the efficiency of using deep learning models in solving Arabic multi-label text classification problem. Multilayer Perceptron (MLP) and Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) have been used; we build two models via python. All data has been cleaned to improve the quality of experimental data. The result of test data in LSTM was 82.03 whereas in the MLP model was 80.37, and both models were evaluated using F1 score.

Detecting fake news is a critical challenge in natural language processing (NLP), demanding solutions that balance accuracy, interpretability, and computational efficiency. Despite advances in NLP, systematic empirical benchmarks that directly compare both classical and deep models—across varying input richness and with careful attention to interpretability and computational tradeoffs—remain underexplored. In this study, we systematically evaluate the mathematical foundations and empirical performance of five representative models for automated fake news classification: three classical machine learning algorithms (Logistic Regression, Random Forest, and Light Gradient Boosting Machine) and two state-of-the-art deep learning architectures (A Lite Bidirectional Encoder Representations from Transformers—ALBERT and Gated Recurrent Units—GRUs). Leveraging the large-scale WELFake dataset, we conduct rigorous experiments under both headline-only and headline-plus-content input scenarios, providing a comprehensive assessment of each model’s capability to capture linguistic, contextual, and semantic cues. We analyze each model’s optimization framework, decision boundaries, and feature importance mechanisms, highlighting the empirical tradeoffs between representational capacity, generalization, and interpretability. Our results show that transformer-based models, especially ALBERT, achieve state-of-the-art performance (macro F1 up to 0.99) with rich context, while classical ensembles remain viable for constrained settings. These findings directly inform practical fake news detection.

Depression detection in low-resource languages, such as Bangla, is an emerging challenge in mental health research due to the limited availability of labeled datasets and NLP resources. Bangla, the seventh most spoken language globally, is used by over 230 million people, yet it remains underrepresented in natural language processing (NLP) research compared to high-resource languages like English. In this study, we present a Bangla depression detection dataset collected from social media comprising 18,612 text samples, with 8556 labeled as depressive and the remaining as non-depressive. To effectively classify depressive content, we explore multiple deep learning architectures, including Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), Gated Recurrent Unit (GRU), and Bidirectional GRU (BiGRU). Additionally, we evaluate state-of-the-art transformer-based models such as BanglaBERT, BanglaBERT-Base, ELECTRA, RoBERTa, and BERT. Machine learning models are also employed as baselines for comparison. Our experimental results show that transformer-based models outperform both deep learning and machine learning approaches, with BanglaBERT achieving the highest accuracy of 89.23%, followed closely by BiLSTM with 88%. These findings highlight the effectiveness of transformer-based architectures in depression detection for Bangla text, contributing to improved mental health analysis in low-resource language settings. This study emphasizes the need for further NLP advancements in Bangla to develop AI-driven solutions for early depression detection and intervention.

Leaves are one of the plant components that play a crucial role in plant productivity. Leaf disease in lemon plants can negatively impact both productivity and plant health. Therefore, a computer-based system for auto-matic classification was developed to efficiently and accurately identify dis-eases in lemon leaves. The automatic disease classification process in lemon leaves plays an important role for farmers, supporting timely and accurate decision-making. This study compared several deep learning and transformer models for disease classification of lemon leaves based on their images. The main objective of this study was to determine the performance of each deep learning and transformer architecture for disease classification in lemon leaves using image-based features. In evaluating various experiments, two deep learning models performed well: InceptionV3 and Xception, with accu-racies of 99.85% and 99.63%, respectively, and minimum loss values of 0.017% and 0.0018%, respectively. Then, the transformer model using the Vision Transformer architecture outperformed the two best deep learning models, achieving 99.89% accuracy and a loss of 0.0013. Overall, the experiment comparing the deep learning model and the transformer shows that the Vision Transformer outperforms the CNN. Consequently, the Vision Trans-former presents strong potential as an effective and robust approach for im-age-based leaf disease detection in lemon plants.