Adaptive Mapping for Data Orchestration in Reconfigurable CNN Processors via Deep Reinforcement Learning

In this paper, we provide expressions for the secrecy outage probability (SOP) for suboptimal and optimal opportunistic scheduling schemes in a reconfigurable intelligent surface (RIS) aided single antenna system with multiple eavesdroppers in approximate closed form. A suboptimal scheduling (SS) scheme is analyzed, which is used when the channel state information (CSI) of the eavesdropping links is unavailable, and the optimal scheduling (OS) scheme is also analyzed, which is used when the global CSI is available. For each scheme, we provide a simplified expression for the SOP in the high signal-to-noise ratio (SNR) regime to demonstrate its behavior as a function of the key system parameters. At high SNR, the SOP saturates to a constant level which decreases exponentially with the number of RIS elements in the SS scheme and with the product of the number of RIS elements and the number of users in the OS scheme. We also show that the derived SOP of the SS scheme can directly provide the SOP for the best antenna-user pair scheduling scheme in a multiple antenna system. We compare the performance of the opportunistic user scheduling schemes with that of a non-orthogonal multiple access (NOMA) based scheduling scheme which chooses a pair of users in each time slot for scheduling and we show that the opportunistic schemes outperform the NOMA-based scheme. We also derive a closed-form expression for the SOP of a decode-and-forward (DF) relay-aided scheduling scheme in order to compare it with that of the RIS-aided system. It is found that the RIS-aided system outperforms the relay-aided system when the number of RIS elements is sufficiently large. An increased number of RIS elements is required to outperform the relay-aided system at higher operating frequencies.

Secrecy outage analysis of multi-user multi-eavesdropper cellular networks in the face of cochannel interference

This paper investigates the coexistence of the New Radio-based access to Unlicensed spectrum (NR-U) network and the Wireless Gigabit (WiGig) network in unlicensed millimeter-wave (mmWave) bands. To enable the NR-U network to achieve equitable and harmonious spectrum sharing with WiGig systems, we develop two new classes of user equipment (UE) scheduling schemes by exploiting the deep reinforcement learning (DRL) technique. Specifically, we first propose the distributed deep reinforcement learning scheduling (DeepDS) scheme, wherein multiple deep neural networks (DNNs) are used to make decisions for different panels in an independent fashion. Thereafter, to reduce the computational cost of adopting multiple DNNs, we design the centralized deep reinforcement learning scheduling (DeepCS) scheme that introduces the shared DNN framework to perform decisions for all panels in parallel at one time. The objective of both DeepDS and DeepCS is to maximize the total data rate of the NR-U network with as little interference to WiGig systems as possible, while satisfying the quality of service (QoS) requirement for each UE. We first formulate this problem into the constrained Markov decision process framework. To address the multi-constraint issue, we put forth a new DRL algorithm that incorporates the Lagrangian primal-dual optimization into the deep Q-network framework, referred to as adaptive multi-constraint deep Q-network (AMC-DQN). With AMC-DQN, both DeepDS and DeepCS can achieve their goals even without acquiring prior operations about the WiGig network. Simulation results show that compared with the state-of-the-art omniLBT and dirLBT, both DeepDS and DeepCS yield significant performance benefits in terms of the total network data rate. We also demonstrate the ability of DeepDS and DeepCS to satisfy the QoS requirements of different UEs and their robustness against various simulation setups. Furthermore, compared with DeepDS, DeepCS can save a large amount of computational cost although at the expense of a slightly lower data rate.

The success of the smart grid majorly depends on the advanced communication architectures. An advanced smart grid network should satisfy the future demands of the electric systems in terms of reliability and latency. The latest 4th-generation (4G) wireless technology, the 3rd Generation Partnership Project (3GPP) Long Term Evolution (LTE), is a promising choice for smart grid wide area networks (WAN), due to its higher data rates, lower latency and larger coverage. However, LTE is not a dedicated technology invented for smart grid, and it does not provide Quality of Service (QoS) guarantee to the smart grid applications. In this paper, we propose an optimal LTE uplink scheduling scheme to provide scheduling timeguarantee at the LTE base station for different class of traffic, with a minimal number of total resource blocks. A lightweight heuristic algorithm is proposed to obtain the optimal allocation of resource blocks for each class of traffic. In the simulation, we compare the proposed optimal scheduling scheme and two existing scheduling schemes, the Large-Metric-First scheduling scheme and the Guaranteed Bit Rate (GBR) /Non-GBR scheduling scheme. The comparison results demonstrate that the proposed optimal scheduling can use less resource blocks to satisfy the scheduling time requirements than the other two existing scheduling schemes.

Electric vehicle clusters scheduling strategy considering real-time electricity prices based on deep reinforcement learning

At present, due to the errors of wind power, solar power and various types of load forecasting, the optimal scheduling results of the integrated energy system (IES) will be inaccurate, which will affect the economic and reliable operation of the integrated energy system. In order to solve this problem, a day-ahead and intra-day optimal scheduling model of integrated energy system considering forecasting uncertainty is proposed in this paper, which takes the minimum operation cost of the system as the target, and different processing strategies are adopted for the model. In the day-ahead time scale, according to day-ahead load forecasting, an integrated demand response (IDR) strategy is formulated to adjust the load curve, and an optimal scheduling scheme is obtained. In the intra-day time scale, the predicted value of wind power, solar power and load power are represented by fuzzy parameters to participate in the optimal scheduling of the system, and the output of units is adjusted based on the day-ahead scheduling scheme according to the day-ahead forecasting results. The simulation of specific examples shows that the integrated demand response can effectively adjust the load demand and improve the economy and reliability of the system operation. At the same time, the operation cost of the system is related to the reliability of the accurate prediction of wind power, solar power and load power. Through this model, the optimal scheduling scheme can be determined under an acceptable prediction accuracy and confidence level.

Source-load uncertainty-based multi-objective multi-energy complementary optimal scheduling

The boom of video streaming industry has resulted in the increasing demands for transcoding services from heterogeneous users. Recent advances of blockchain technology allow some startups to realize decentralized collaborative transcoding through device-to-device (D2D) networks, where a group of transcoders are selected to perform transcoding cooperatively. For the blockchain-enabled D2D transcoding systems, it's imperative to jointly design transcoder selection, task scheduling and resource allocation schemes in order to provide efficient and trustworthy transcoding services. In this paper, viewing the involved multi-dimensional complex factors and channel fluctuation, we propose a novel deep reinforcement learning (DRL) based transcoder selection framework for blockchain enabled D2D transcoding systems where both the platform dynamics and channel statistics are captured. To reduce the action space size, we adopt a two-stage decision approach to first select the transcoders through a normal DRL based framework and then obtain the optimal task scheduling, power control, and resource allocation scheme by solving a stochastic optimization problem with the constrained stochastic successive convex approximation (CSSCA) approach. Simulation results show that our proposed framework can achieve high transcoding revenue while meeting the quality of service (QoS) requirements, and it can well handle dynamic cases.

Distributed Opportunistic Scheduling in Power Systems – An Optimal Stopping Approach

Software project scheduling is essentially a kind of project scheduling problem with limited human resources. During the development process of a software product, reworking the completed projects, reassessing the workload, and changing the number of employees or their skills are the frequently occurring dynamic issues having direct influences on designing a scheduling scheme. Taking the development cost and duration, the robustness, and the stability of the scheduling schemes as the objective functions, software project scheduling is modeled as a dynamic four‐objective optimization problem. The various programming habits among the employees form the specific constraints for the reworking tasks, and the skills of the employees vary due to the effects of learning and forgetting. To solve this problem, an improved multiobjective firework algorithm with a novel explosion operator and reservation strategy is incorporated with rescheduling methods to fully guide the evolution by using the historical evolutionary knowledge. The experimental results indicate that the proposed method has better scheduling performance, and the optimal scheduling schemes have better robustness and stability.

This study focuses on the challenges of aviation maintenance technician (AMT) scheduling and constructs a model based on personnel satisfaction and the parallel execution of aircraft maintenance tasks. To obtain the scheduling scheme from the constructed NP‐hard model, an interactive multi swarm bacterial foraging optimization (IMSBFO) algorithm is proposed using multi‐swarm coevolu tion, structural recombination, and three informa tion interactive mechanisms among individuals. Moreover, considering the distributed feature of the AMT scheduling problem, a specific mechanism is designed to convert continuous solution to a binary AMT scheduling scheme. Finally, a series of com parative experiments highlight the efficiency and superiority of our proposed IMSBFO algorithm, and the optimal scheduling scheme owns the delicate balance between the work and rest time.

Residential power scheduling for demand response in smart grid

With the rapid development of semiconductor manufacturing, customers' demand for on-time delivery rate (ODR) makes scheduling strategies face new challenges. In order to meet customers' delivery requirements, scheduling strategies generally need to comprehensively consider remaining cycle time (CT), ODR, movement (MOV) speed and machine load balancing. In order to solve these problems, this paper proposed a scheduling strategy of semiconductor production lines with remaining cycle time prediction. Firstly, gather features related to performance index and then filtrate them through dimension reduction method. Secondly, use the above feature subset to build remaining cycle time prediction model by random forest algorithm. Next, design the scheduling strategy of semiconductor production lines with remaining cycle time prediction. Finally, make simulation experiments to verify the effectiveness of the proposed scheduling strategy. Simulation results show that the proposed scheduling strategy can improve the mean CT, throughput (TH), machine utilization time (MUT) and ODR in different extant.

In the domain of continuous control, deep reinforcement learning (DRL) demonstrates promising results. However, the dependence of DRL on deep neural networks (DNNs) results in the demand for extensive data and increased computational cost. To address this issue, a novel hybrid actor-critic reinforcement learning (RL) framework is introduced. The proposed framework integrates the broad learning system (BLS) with DNN, aiming to merge the strengths of both distinct architectural paradigms. Specifically, the critic network employs BLS for rapid value estimation via ridge regression, while the actor network retains the DNN structure to optimize policy gradients. This hybrid design is generalizable and can enhance existing actor-critic algorithms. To demonstrate its versatility, the proposed framework is integrated into three widely used actor-critic algorithms-deep deterministic policy gradient (DDPG), soft actor-critic (SAC), and twin delayed DDPG (TD3), resulting in BLS-augmented variants. The experimental results reveal that all BLS-enhanced versions surpass their original counterparts in terms of training efficiency and accuracy. These improvements highlight the suitability of the proposed framework for real-time control scenarios, where computational efficiency and rapid adaptation are critical.

Precise trajectory tracking is crucial for Autonomous Underwater Vehicles (AUV) to handle complex missions effectively. However, strong ocean currents make it difficult to balance system robustness with smooth control signals. Existing research has primarily focused on fixed-gain sliding mode control strategies, utilizing high switching gains to cover the upper bounds of disturbances. However, these approaches depend heavily on conservative parameter settings, leading to severe chattering and performance degradation when the AUV faces time-varying marine environments. In this study, an approach employing an adaptive double-layer integral sliding mode controller based on Deep Reinforcement Learning is proposed for the AUV trajectory tracking problem. This method introduces the Deep Deterministic Policy Gradient algorithm to construct an agent capable of the online optimization and output of three core control parameters according to real-time system states. The superior performance and adaptability originate from a composite reward function designed to balance tracking accuracy with output smoothness, guiding the agent to learn a state-dependent optimal gain scheduling strategy. Based on the Lyapunov theory combined with Barbalat's lemma, the stability of the non-autonomous time-varying system is strictly proved. Simulation results demonstrate that the proposed method effectively suppresses the chattering problem caused by high gains while ensuring robustness, exhibiting comprehensive performance significantly superior to that of fixed-gain and PD sliding mode controllers.