Automatic Inside Point Localization with Deep Reinforcement Learning for Interactive Object Segmentation.

In this paper, we address the difficulty of solving large-scale multi-dimensional knapsack instances (MKP), presenting a novel deep reinforcement learning (DRL) framework. In this DRL framework, we train different agents compatible with a discrete action space for sequential decision-making while still satisfying any resource constraint of the MKP. This novel framework incorporates the decision variable values in the 2D DRL where the agent is responsible for assigning a value of 1 or 0 to each of the variables. To the best of our knowledge, this is the first DRL model of its kind in which a 2D environment is formulated, and an element of the DRL solution matrix represents an item of the MKP. Our framework is configured to solve MKP instances of different dimensions and distributions. We propose a K-means approach to obtain an initial feasible solution that is used to train the DRL agent. We train four different agents in our framework and present the results comparing each of them with the CPLEX commercial solver. The results show that our agents can learn and generalize over instances with different sizes and distributions. Our DRL framework shows that it can solve medium-sized instances at least 45 times faster in CPU solution time and at least 10 times faster for large instances, with a maximum solution gap of 0.28% compared to the performance of CPLEX. Furthermore, at least 95% of the items are predicted in line with the CPLEX solution. Computations with DRL also provide a better optimality gap with respect to state-of-the-art approaches.

With the rapid development of smart grid and renewable energy, demand response mechanism has become a key means to balance microgrid supply and demand and improve energy efficiency. As the traditional power demand response algorithm is difficult to deal with the uncertainty of power demand and the adverse effects caused by customers, this paper proposes a power demand response algorithm based on deep reinforcement learning. Firstly, we study and design a price -based layered electricity energy demand response model. Then, the dynamic optimal pricing decision of power trading market is described as Markov decision process, and the learning mechanism under the framework of deep reinforcement learning is expounded. Finally, a solution algorithm based on deep reinforcement learning is designed. The simulation results show that the proposed algorithm can adjust the electricity price adaptively according to the load demand, which can reduce the cost of users by about 17% and reduce the peak load demand.

Optimal scheduling of island integrated energy systems considering multi-uncertainties and hydrothermal simultaneous transmission: A deep reinforcement learning approach

Effective ways to optimise real-time pump scheduling to maximise energy efficiency are being sought to meet the challenges in the energy market. However, the considerable number of evaluations of popular optimisation methods based on metaheuristics cause significant delays for real-time pump scheduling, and the simplification of traditional deterministic methods may introduce bias towards the optimal solutions. To address these limitations, an exploration-enhanced deep reinforcement learning (DRL) framework is proposed to address real-time pump scheduling problems in water distribution systems. The experimental results indicate that E-PPO can learn suboptimal scheduling policies for various demand distributions and can control the application time to 0.42 s by transferring the online computation-intensive optimisation task offline. Furthermore, a form of penalty of the tank level was found that can reduce energy costs by up to 11.14% without sacrificing the water level in the long term. Following the DRL framework, the proposed method makes it possible to schedule pumps in a more agile way as a timely response to changing water demand while still controlling the energy cost and level of tanks.

To address the static limitations of current career assessment methods and the lack of a long-term perspective in planning strategies, this study constructs an intelligent analysis framework named Deep learning-driven Value-added Career Analytics Model (DV-CAM). Integrating Deep Learning (DL) with value-added assessment, the framework aims to realize dynamic and accurate evaluation of individual professional competencies and generate personalized career planning paths that maximize the benefits of long-term development. Specifically, DV-CAM first leverages Bidirectional Encoder Representations from Transformers (BERT) and Long Short-Term Memory (LSTM) to extract and dynamically update individual competency profiles from multi-source time-series data. Second, it designs a value-added evaluation index system covering the absolute level of competency, growth slope, growth curvature and stability to quantitatively diagnose development potential. Finally, under the framework of Deep Reinforcement Learning (DRL), this study takes competency status, value-added potential and job requirements as joint state inputs, and solves for the optimal sequence of career development actions by optimizing long-term cumulative rewards. Experiments based on the public Occupational Information Network (O*NET) dataset show that DV-CAM achieves a mean squared error of 0.0312 and a mean absolute error of 0.1289 for competency assessment on the test set. In planning tasks, its long-term cumulative reward and target job matching degree improvement rate reach 185.3% and 42.7% respectively, both outperforming various benchmark models. The results demonstrate that DV-CAM can diagnose competency status more accurately, and effectively guide individuals to achieve long-term career development goals through its forward-looking planning strategies. By combining the concept of value-added assessment with advanced deep learning technologies, this study provides a data-driven, dynamically perceivable and long-term oriented new paradigm for the fields of career planning and competency assessment.

Task assignment is a challenging problem in multiple unmanned aerial vehicle (UAV) missions. In this paper, we focus on the task assignment problem for a UAV swarm saturation attack, in which a deep reinforcement learning (DRL) framework is developed. Specifically, we first construct a mathematical model to formulate the task assignment problem for a UAV swarm saturation attack and consider it as a Markov Decision Process (MDP). We then design a policy neural network using the attention mechanism. We also propose a training algorithm based on the policy gradient method so that our agent can learn an effective task assignment policy. The experimental results have shown that our DRL method can generate high-quality solutions for different problem scales, which meets the requirements of real-time and flexibility in the actual situation.

Manually designing an effective and efficient controller for complex mechanics, such as bio-inspired robots or underactuated mechanical system, typically are very difficult. It requires precise motion planning and dynamic control. Reinforcement learning or genetic algorithm based learning methods suffers from representing the high dimensional models. The combination of deep learning and reinforcement learning provide a feasible way to handle such difficulties. However, priori-less searching sometimes tends to be low efficient and usually finds the “mechanic” solution instead of the “natural” one. In this paper, the traditional nonlinear control concept is integrated into the deep reinforcement learning (DRL) framework. The whole process is implemented on the brachiation control problem of a two link bio-primate robot. Deep Deterministic Policy Gradient (DDPG) is used to search for the optimal control policy. The searching process is realized by interacting with the dynamic model instead of real robot. The energy based planning and control concept is adopted, which utilize the fact that when the shoulder joint angle is fixed, energy of the whole system keeps constant. By regulating the angle and energy, the robot can be restricted on a particular trajectory. The energy concept is encoded within the reward function and trained in the Gym environment. For varying targets point-to-point control, the network structure is also modified to accept the target coordinates. Effectiveness of the proposed methods are verified by simulation and experimental results.

A multi-objective deep reinforcement learning framework

Non-terrestrial network (NTN) communication is included in the 3GPP standard because of its excellent features such as resistance to ground physical attacks and wide coverage. Since the available power and storage resources are limited on high throughput satellites (HTSs), optimizing the resource allocation can greatly improve the performance of the HTS based communication system. Notably, weather and other factors make the channel status between the satellite and the terrestrial base stations constantly change. In this paper, we first exploit the model-free feature of reinforcement learning to formulate the aforementioned dynamic and unpredictable channel conditions into the power allocation problem in HTS systems. Due to the complexity of environment state and power allocation, a deep neural network is introduced to replace the Q table, and a deep reinforcement learning framework is built. Furthermore, a power allocation algorithm based on a deep reinforcement learning framework is presented. Finally, the simulation results show that the proposed algorithm is superior to existing algorithms in terms of the long-term system throughput performance, and has better adaptability to dynamic channel environment, thereby improving network performance.

This paper proposes a deep reinforcement learning (DRL) framework that learns a reusable policy to find the optimal placement of decoupling capacitors (decaps) on power distribution network (PDN) with an arbitrary probing port. The proposed DRL framework trains a policy parameterized by pointer network, which is a sequence-to-sequence neural network, based on REINFORCE algorithm. The policy finds the positional combination of a pre-defined number of decaps that best suppresses self-impedance of a given probing port on PDN with randomly assigned keep-out regions. Verification was done by allocating 20 decaps on ten randomly generated test sets with an arbitrary probing port and randomly selected keep-out regions. Performance of the policy generated by the proposed DRL framework was evaluated based on the magnitude of probing port self-impedance suppression followed by decap placement over 434 frequencies between 100MHz and 20GHz. The policy generated by the proposed framework achieves greater impedance suppression with fewer samples in comparison to random search heuristic method.

Internet of Battlespace Things (IoBT) deployments in adversarial environments face critical challenges in network resource management, where rapid environmental changes and dynamic threats render traditional optimization approaches inadequate. The inherently volatile nature of these environments, where network conditions can change within seconds, necessitates algorithms that prioritize speed over perfect optimization to maintain operational effectiveness. This paper presents a novel Graph Neural Network (GNN) based Deep Reinforcement Learning (DRL) framework specifically designed for combinatorial task admission and routing optimization in dynamic IoBT networks. Our approach integrates Graph Attention Networks (GATs) for capturing network topology dependencies, Deep Sets encoders for permutation-invariant task processing, Adaptive Path GNNs for learning path representations, and statistical feature encoders that complement learned embeddings with interpretable routing heuristics. The system formulates network optimization as a Markov Decision Process, enabling real-time decision-making that maximizes task utility while respecting capacity constraints and adapting to topology changes. Comprehensive experimental evaluation across multiple network scenarios demonstrates that our Deep Q-Network (DQN) agent consistently outperforms greedy baselines by 5-58%, achieving near-optimal utility with sub-second inference times compared to Mixed Integer Programming solvers that require hundreds of seconds. The framework shows strong generalization capabilities, with agents trained on smaller task sets effectively scaling to larger workloads, and exhibits superior performance when trained under dynamic conditions rather than static environments.

In this paper, we propose a two-stage Deep Reinforcement Learning (DRL) framework for the automatic arrangement of buildings in urban planning, with a focus on satisfying critical regulatory constraints: sunshine duration and inter-building distance. To overcome the computational bottleneck of high-fidelity sunshine simulation during DRL training, we first introduce a Convolutional Auto-Encoder (CAE) model that serves as a fast and accurate surrogate for predicting sunshine distribution across a building layout. This CAE model, trained on data generated from Autodesk Revit, achieves near real-time prediction (∼0.015 s per layout) with an average error of less than 3 min, enabling its seamless integration into the iterative RL loop. Subsequently, we formulate the sequential placement of buildings as a Markov Decision Process (MDP) and employ a Deep Q-Network (DQN) to learn the optimal placement policy. The DQN agent interacts with a planning environment, where its actions (moving a building in discrete directions) are rewarded based on the satisfaction of distance constraints and the avoidance of low-sunshine zones identified by the CAE. Our integrated CAE-DQN system automates the generation of compliant building layouts. Experimental evaluation demonstrates that our method significantly outperforms baseline approaches (Random Placement, Greedy Heuristic, and Genetic Algorithm) in terms of success rate (100%), convergence speed, and layout compactness. The proposed framework not only ensures hard constraint satisfaction but also provides a practical and scalable tool for rapid prototype generation in the conceptual design phase of urban planning.

We present a hierarchical deep reinforcement learning (DRL) framework with prominent sampling efficiency and sim-to-real transfer ability for fast and safe navigation: the low-level DRL policy enables the robot to move towards the target position and keep a safe distance to obstacles simultaneously; the high-level DRL policy is supplemented to further enhance the navigation safety. We select a waypoint located on the path from the robot to the ultimate goal as the sub-goal to reduce the state space and avoid sparse reward. Moreover, the path is generated based on either a local or a global map, which can significantly improve the sampling efficiency, safety, and generalization ability of the proposed DRL framework. Additionally, a target-directed representation for the action space can be derived based on the sub-goal to improve the motion efficiency and reduce the action space. In order to demonstrate the eminent sampling efficiency, motion performance, obstacle avoidance, and generalization ability of the proposed framework, we implement sufficient comparisons with the non-learning navigation methods and DRL-based baselines, with videos, data, code, and other supplemental material shown on our website<xref ref-type="fn" rid="fn1">¹</xref><fn id="fn1"><label>¹</label> https://github.com/zw199502/RL <inline-formula><tex-math notation="LaTeX">$\_$</tex-math></inline-formula> navigation </fn>.

With The advent of the Internet of Things (IoT) and its use cases there is a necessity for improved latency which has led to edgecomputing technologies. IoT applications need a cloud environment and appropriate scheduling based on the underlying requirements of a given workload. Due to the mobility nature of IoT devices and resource constraints and resource heterogeneity, IoT application tasks need more efficient scheduling which is a challenging problem. The existing conventional and deep learning scheduling techniques have limitations such as lack of adaptability, issues with synchronous nature and inability to deal with temporal patterns in the workloads. To address these issues, we proposed a learning-based framework known as the Deep Reinforcement Learning Framework (DRLF). This is designed in such a way that it exploits Deep Reinforcement Learning (DRL) with underlying mechanisms and enhanced deep network architecture based on Recurrent Neural Network (RNN). We also proposed an algorithm named Reinforcement Learning Dynamic Scheduling (RLbDS) which exploits different hyperparameters and DRL-based decision-making for efficient scheduling. Real-time traces of edge-cloud infrastructure are used for empirical study. We implemented our framework by defining new classes for CloudSim and iFogSim simulation frameworks. Our empirical study has revealed that RLbDS out performs many existing scheduling methods.

The proliferation of the Internet of Things (IoT) technology for wide range of wireless applications increase raw data, leads to spectrum scarcity, and also burdens available spectrum resources. Cognitive Radio (CR) is one of the key enablers to build future IoT systems, which has the ability to exploit dynamic networking characteristics to optimize network performance. One of the critical challenges is to obtain perfect Channel State Information (CSI) to efficiently allocate the transmission power and frequency resources among IoT nodes. In this paper, we focus on resource management optimization problem which is transformed into Markov Decision Process (MDP) and solved with proposed Deep Recurrent Reinforcement Learning (DRRL) based scheme with the consideration of energy-efficiency to deal with partial observability from limited information. Furthermore, to heighten the performance of proposed scheme, cooperative framework is introduced with Gated Recurrent Unit (GRU) layer to improve the resource allocation policy by accessing the historical information. Finally, extensive simulations corroborate the superiority of proposed scheme in terms of faster convergence and higher reward. The results also show that CSI imperfections is non-negligible, thus highlights the importance of robust and intelligent design that maintains users’ QoS.