Adaptive Object Exploration and Mapping through Meta-Reinforcement Learning

In modern industrial and logistics environments, the rapid expansion of fast delivery services has heightened the demand for storage systems that combine high efficiency with increased density. Multi-deep autonomous vehicle storage and retrieval systems present a viable solution for achieving greater storage density. However, these systems encounter significant challenges during retrieval operations due to lane blockages. A conventional approach to mitigate this issue involves storing items with homogeneous characteristics in a single lane, but this strategy restricts the flexibility and adaptability of multi-deep storage systems. Building on this background, this work presents a deep reinforcement learning-based framework to optimize the retrieval process in multi-deep storage systems with heterogeneous item configurations. Each item is associated with a specific due date, and objective function is total tardiness minimization. To effectively capture the system’s topology, we introduce a graph-based state representation that integrates both item attributes and the local topological structure of the multi-deep warehouse. For processing this representation, we design a novel neural network architecture that combines a Graph Neural Network (GNN) with a Transformer model. The GNN encodes topological and item-specific information into embeddings for all directly accessible items, while the Transformer maps these embeddings into global priority assignments. The Transformer’s strong generalization capability further allows our approach to be applied to storage systems with diverse layouts. Extensive numerical experiments, including comparisons with heuristic methods, demonstrate the superiority of the proposed neural network architecture and the effectiveness of the trained agent in optimizing retrieval tardiness.

Human operators have the trend of increasing physical and mental workloads when performing teleoperation tasks in uncertain and dynamic environments. In addition, their performances are influenced by subjective factors, potentially leading to operational errors or task failure. Although agent-based methods offer a promising solution to the above problems, the human experience and intelligence are necessary for teleoperation scenarios. In this paper, a truncated quantile critics reinforcement learning-based integrated framework is proposed for human–agent teleoperation that encompasses training, assessment and agent-based arbitration. The proposed framework allows for an expert training agent, a bilateral training and cooperation process to realize the co-optimization of agent and human. It can provide efficient and quantifiable training feedback. Experiments have been conducted to train subjects with the developed algorithm. The performances of human–human and human–agent cooperation modes are also compared. The results have shown that subjects can complete the tasks of reaching and picking and placing with the assistance of an agent in a shorter operational time, with a higher success rate and less workload than human–human cooperation.

Safe and efficient navigation in dynamic pedestrian environments remains a significant challenge for mobile robots, particularly due to rapidly changing scenarios and complex multi-agent interactions. This paper proposes CBRNPPO, a deep reinforcement learning-based navigation framework designed to address these challenges. The framework incorporates a ResNet-based feature extractor enhanced with the Convolutional Block Attention Module (CBAM) to improve both perception and decision-making. By fusing multimodal inputs—including LiDAR scans, pedestrian velocity maps, and target direction vectors—the model effectively captures rich semantic representations of dynamic obstacles. The CBAM module adaptively emphasizes critical spatial and channelwise features, enhancing robustness in densely populated environments. Experimental results demonstrate that in scenarios with 35 pedestrians, CBRN-PPO achieves a 93% success rate in obstacle avoidance, outperforming the A1-RD baseline by 15%. Furthermore, it improves path efficiency by 38% and average navigation speed by 34%. These results highlight the effectiveness and robustness of CBRN-PPO as a navigation solution for autonomous mobile robots operating in complex dynamic environments.

Automated guided vehicle dispatching and routing integration via digital twin with deep reinforcement learning

Brain tumour is one of the most aggressive malignant cancers and early-accurate diagnosis may increase the changes in successful treatment and subsequently improvement in patient's survival. Magnetic Resonance Imaging (MRI) is currently the gold standard diagnostic technique, but manual interpretation becomes slow, unreliable and radiologist skill-dependent when applied to large numbers of images. To overcome these limitations, in this work, we propose an automated deep learning-based brain tumor detection framework using transfer learning with VGG16 and VGG19 models. Both the models were pre-trained and fine-tuned using a brain MRI dataset, and their performance was measured in terms of accuracy, loss, precision, recall and F1-score. The experimental results show a trade-off between performance and efficiency: VGG19 achieves better accuracy at the cost of slower convergence time and heavier computation, whereas VGG16 provides faster convergence rate with less cost. In the interest of transparency and clinical confidence in these predictions, we used explainable AI (XAI) methods – namely Grad-CAM, LIME, and SHAP – to generate visual level as well as feature-level explanations of how each model makes its predictions. The deployed models along with the XAI outputs were implemented with a web-based framework written in Flask, which allows real-time MRI upload, prediction and visualization of interpretability map. We showed the advantage of integrating deep learning, explainability, as well as deployment for robust and interpretable brain tumor diagnostic assistance.

This research work aims to develop a deep learning-based crop classification framework for remotely sensed time series data. Tobacco is a major revenue generating crop of Khyber Pakhtunkhwa (KP) province of Pakistan, with over 90% of the country’s Tobacco production. In order to analyze the performance of the developed classification framework, a pilot sub-region named Yar Hussain is selected for experimentation work. Yar Hussain is a tehsil of district Swabi, within KP province of Pakistan, having highest contribution to the gross production of the KP Tobacco crop. KP generally consists of a diverse crop land with different varieties of vegetation, having similar phenology which makes crop classification a challenging task. In this study, a temporal convolutional neural network (TempCNNs) model is implemented for crop classification, while considering remotely sensed imagery of the selected pilot region with specific focus on the Tobacco crop. In order to improve the performance of the proposed classification framework, instead of using the prevailing concept of utilizing a single satellite imagery, both Sentinel-2 and Planet-Scope imageries are stacked together to assist in providing more diverse features to the proposed classification framework. Furthermore, instead of using a single date satellite imagery, multiple satellite imageries with respect to the phenological cycle of Tobacco crop are temporally stacked together which resulted in a higher temporal resolution of the employed satellite imagery. The developed framework is trained using the ground truth data. The final output is obtained as an outcome of the SoftMax function of the developed model in the form of probabilistic values, for the classification of the selected classes. The proposed deep learning-based crop classification framework, while utilizing multi-satellite temporally stacked imagery resulted in an overall classification accuracy of 98.15%. Furthermore, as the developed classification framework evolved with specific focus on Tobacco crop, it resulted in best Tobacco crop classification accuracy of 99%.

Along with the various technological advancements, the next generation vehicular networks such as the Internet of Vehicles (IoV) also bring in various cybersecurity challenges. To effectively address these challenges, in addition to the existing authentication techniques, there is also a need for identification of the misbehaving entities in the network. This letter proposes a deep learning-based classification framework to identify potential misbehaving vehicles before the communication requests from the On Board Units (OBUs) of the vehicles can be entertained by the network infrastructure such as the Road Side Units (RSUs). The evaluated metrics demonstrate the performance of the proposed classification approaches.

DL-MSTO+: A deep learning-based multi-scale topology optimization framework via positive definiteness ensured material representation network

Deep learning-based transient stability assessment framework for large-scale modern power system

Alloy gradient-grained structures (represented by copper as a typical single-phase face-centered cubic (FCC) metal), known for their superior mechanical properties such as enhanced strength, ductility, and fatigue resistance, have become increasingly important in aerospace and automotive industries. These alloys are often fabricated using advanced processing techniques such as laser welding, electron beam melting, and controlled cooling, which induce spatial gradients in grain size and optimize material properties by overcoming the traditional strength–ductility trade-off. In this study, a deep learning-based inversion framework combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks is proposed to efficiently predict key constitutive parameters, such as the initial critical resolved shear stress and hardening modulus, in alloy gradient-grained structures. The model integrates spatial features extracted from strain-field sequences and grain morphology images with temporal features from loading sequences, providing a comprehensive solution for path-dependent mechanical behavior modeling. Trained on high-fidelity Crystal Plasticity Finite Element Method (CPFEM) simulation data, the proposed framework demonstrates high prediction accuracy for the constitutive parameters. The model achieves an error margin of less than 5%. This work highlights the potential of deep learning techniques for the efficient and physically consistent identification of constitutive parameters in alloy gradient-grained structures, offering valuable insights for alloy design and optimization.

In this article, we present a novel deep learning-based group contribution framework for the targeted design of ionic liquids (ILs). This computational framework can expedite and improve the process of finding desirable molecular structures of IL via accurate property predictions in a data-driven manner. Our proposed framework consists of two essential steps: establishing a correlation between IL viscosity and CO2 solubility by merging two deep learning models (DNN-GC and ANN-GC) and utilizing this correlation to identify the optimal IL structure with maximal CO2 absorption capacity. Our model achieves high accuracy with R2 values of 95%, 94.2%, and 96.4% for DNN-GC, ANN-GC, and DNN-ANN-GC, respectively. Correlation results align with the experimental data, affirming the applicability of our framework. Finally, the algorithm is employed in a CO2 capture case study to generate and select the best-performing novel ILs, which exhibit behavior consistent with established ILs in the literature.

The emergence of deep learning has paved to solve many problems in the real world. COVID-19 pandemic, since the late 2019, has been affecting lives of people across the globe. Chest CT scan images are used to detect it and know its severity in patients. The problem with many existing solutions in COVID-19 detection using CT scan images is that inability to detect the infection when it is in initial stages. As the infection can exist on varied scales, there is need for more comprehensive approach that can ascertain the disease at all scales. Towards this end, we proposed a deep learning-based framework known as Automated Deep Learning-based COVID-19 Detection Framework (ADL-CDF). It does not need a human medical expert in diagnosis as it is capable of detecting automatically. The framework is assisted by two algorithms that involve image processing and deep learning. The first algorithm known as Region of Interest (ROI)-based Image Filtering (ROI-IF) which analyses given input CT scan images of a patient and discards the ones where ROI is missing. This algorithm minimizes time taken for processing besides reducing false positive rate. The second algorithm is known as Multi-Scale Feature Selection algorithm that fits into the deep learning framework’s pipeline to leverage detection performance of the ADL-CDF. The proposed framework is evaluated against ResNet50V2 and Xception. Our empirical study revealed that our model outperforms the state of the art.

The autonomous driving is increasingly mounting, promoting, and promising the future of fully autonomous and, correspondingly presenting new challenges in the field of safety assurance. The unexpected and sudden lane change are extremely serious causes of traffic accident and, such an accident scheme leads the multiple vehicle collisions. Extensive evaluation of recent crash data we found a crucial indication that autonomous driving systems are most prone to rear-end collision, which is the leading factor of chain crash. Learning based self-developing assessment assists the operators in providing the necessary prediction operations or even replace them. Here we proposed a Reinforcement learning-based conceptual framework for threat assessment system and scrutinize critical situations that leads to multiple vehicle collisions in autonomous driving. This paper will encourage our transport community to rethink the existing autonomous driving models and reach out to other disciplines, particularly robotics and machine learning, to join forces to create a secure and effective system.

Magnetic drug targeting refers to physical manipulation of drugs to specific sites of disease inside the body by means of external magnetic fields. Targeted drug delivery using magnetic nano particles has potential alternative to provide more effective disease therapy as well as to reduce undesired side effects. In this paper, we study an optimized targeting of drug loaded magnetic nano particles for drug delivery system. Using the concept of virtual magnetic field and considering the dominant forces on magnetic nano-particles, optimal path planning scheme is proposed to perform the drug delivery process. Particle swarm optimization algorithm is used to generate an optimal path from all the discovered trajectories. Simulation results showed that proposed algorithm has higher success rate in targeting the drugs with a faster convergence rate towards the optimal solution.

Recently, it is shown that deploying a proper self-supervision is a prospective way to enhance the performance of supervised learning. Yet, the benefits of self-supervision are not fully exploited as previous pretext tasks are specialized for unsupervised representation learning. To this end, we begin by presenting three desirable properties for such auxiliary tasks to assist the supervised objective. First, the tasks need to guide the model to learn rich features. Second, the transformations involved in the self-supervision should not significantly alter the training distribution. Third, the tasks are preferred to be light and generic for high applicability to prior arts. Subsequently, to show how existing pretext tasks can fulfill these and be tailored for supervised learning, we propose a simple auxiliary self-supervision task, predicting localizable rotation (LoRot). Our exhaustive experiments validate the merits of LoRot as a pretext task tailored for supervised learning in terms of robustness and generalization capability. Our code is available at https://github.com/wjun0830/Localizable-Rotation.KeywordsPretext taskAuxiliary self-supervisionSupervised learning