Hierarchical DRL Based Resource Allocation for Latency-Constrained Packet Dropping Rate Optimization in URLLC

The coexistence of Ultra-Reliable and Low-Latency Communications (URLLC) and enhanced Mobile BroadBand (eMBB) brings significant challenges for service pairing and resource allocation in beyond fifth-generation (B5G) wireless networks. To meet the reliability requirement of URLLC services and improve the fairness of eMBB services, we first develop a supervised learning-based resource allocation policy for eMBB services. Then, a two-phase resource allocation framework is proposed for URLLC services: 1) eMBB/URLLC service pairing, and 2) URLLC resource allocation. In the first phase, matching theory pairs eMBB and URLLC services for better fairness. In the second phase, URLLC resource allocation policy is optimized by a constrained unsupervised learning algorithm. Simulation results show that our proposed framework can achieve better trade-offs among fairness, throughput, and reliability compared with two existing baselines. For example, a dynamic proportional fairness algorithm can meet the reliability requirement of URLLC when its average packet arrival rate is below 0.7 packets/mini-slot. The proposed algorithm can support URLLC services with an average packet arrival rate of 1.6 packets/mini-slot

This paper studies the coexistence of enhanced mobile broadband (eMBB) and ultra-reliable and low-latency communication (URLLC) services in a cellular network that is assisted by a reconfigurable intelligent surface (RIS). The system model consists of one base station (BS) and one RIS that is deployed to enhance the performance of both eMBB and URLLC in terms of the achievable data rate and reliability, respectively. We formulate two optimization problems, a time slot basis eMBB allocation problem and a mini-time slot basis URLLC allocation problem. The eMBB allocation problem aims at maximizing the eMBB sum rate by jointly optimizing the power allocation at the BS and the RIS phase-shift matrix while satisfying the eMBB rate constraint. On the other hand, the URLLC allocation problem is formulated as a multi-objective problem with the goal of maximizing the URLLC admitted packets and minimizing the eMBB rate loss. This is achieved by jointly optimizing the power and frequency allocations along with the RIS phase-shift matrix. In order to avoid the violation in the URLLC latency requirements, we propose a novel framework in which the RIS phase-shift matrix that enhances the URLLC reliability is proactively designed at the beginning of the time slot. For the sake of solving the URLLC allocation problem, two algorithms are proposed, namely, an optimization-based URLLC allocation algorithm and a heuristic algorithm. The simulation results show that the heuristic algorithm has a low time complexity, which makes it practical for real-time and efficient multiplexing between eMBB and URLLC traffic. In addition, using only 60 RIS elements, we observe that the proposed scheme achieves around 99.99% URLLC packets admission rate compared to 95.6% when there is no RIS, while also achieving up to 70% enhancement on the eMBB sum rate.

As one of the most important communication scenarios in the coming fifth generation (5G) cellular networks, ultrareliable and low-latency communication (URLLC) is promising to enable real-time wireless control systems. However, one of the biggest challenges is that how to integrate URLLC and control performance together to maximize the overall system performance. In this paper, we investigate the resource allocation for URLLC uplink in real-time wireless control systems. Specifically, we first discuss the relationship between communication and control performance. Based on that, we convert the hybrid co-design problem into a regular wireless resource allocation problem. Then, we propose an iteration algorithm to obtain the optimal wireless resource allocation. Simulation results indicate the performance of our method.

In this paper, we aim to find the global optimal resource allocation for ultra-reliable and low-latency communications (URLLC), where the blocklength of channel codes is short. The achievable rate in the short blocklength regime is neither convex nor concave in bandwidth and transmit power. Thus, a non-convex constraint is inevitable in optimizing resource allocation for URLLC. We first consider a general resource allocation problem with constraints on the transmission delay and decoding error probability, and prove that a global optimal solution can be found in a convex subset of the original feasible region. Then, we illustrate how to find the global optimal solution for an example problem, where the energy efficiency (EE) is maximized by optimizing antenna configuration, bandwidth allocation, and power control under the latency and reliability constraints. To improve the battery life of devices and EE of communication systems, both uplink and downlink resources are optimized. The simulation and numerical results validate the analysis and show that the circuit power is dominated by the total power consumption when the average inter-arrival time between packets is much larger than the required delay bound. Therefore, optimizing antenna configuration and bandwidth allocation without power control leads to minor EE loss.

In this article, we focus on the joint design of channel training and data transmission for secure ultrareliable and low-latency communications (URLLCs) in mission-critical Internet of Things (IoT) scenarios, e.g., intelligent transportation and remote control. Specifically, we consider a multiple-input multiple-output multiantenna eavesdropper (MIMOME) system, and the role of artificial noise (AN) for securing URLLC in this system is studied. In the channel training phase, we use the two-way discriminatory channel estimation (DCE) protocol with AN injection to suppress the channel estimation accuracy at eavesdropper. Meanwhile, the AN-assisted secrecy beamforming scheme is adopted to mask the confidential data signals. Following by the security enhancements above, we provide a quantitative definition of achievable effective secrecy rate (AESR) to measure the performance of our URLLC system with imperfect channel state information (CSI) and short-packet feature. Then, a nonasymptotic closed-form lower bound of AESR is provided to make the numerical calculation tractable, and the asymptotic system performance in the high-SNR regime is also studied to gain a comprehensive insight. Based on the cyclic coordinated search method, we propose an iterative resource allocation algorithm to maximize the AESR of our system, where the blocklength and transmit power assigned to the reverse/forward pilots, confidential data signals, and AN are jointly optimized. In addition, numerical results reveal the AESR performance of our URLLC system for different system parameters, and demonstrate the convergence and superiority of our proposed algorithm.

This paper investigates the resource management problem in multi-carrier rate-splitting multiple access (MC-RSMA) systems with imperfect channel state information (CSI) and successive interference cancellation (SIC) for ultra-reliable and low-latency communications (URLLC) applications. Effective throughput (ET) is employed as the key performance metric in this study to explore the trade-off between the decoding error probability and achievable rate. A mixed-integer non-convex problem is formulated to jointly optimize power allocation, rate control, and user grouping. To tackle this challenge, the achievable ET is approximated using a lower bound, and a decomposition approach is developed to decouple the optimization variables. Specifically, for a given user grouping scheme, an iteration-based concave-convex programming (CCCP) method and an iteration-free lower-bound approximation (LBA) method are proposed for power allocation and rate control. Furthermore, a greedy search-based scheme and a minimum correlation user grouping (MCUG) scheme are designed for the user grouping problem. Simulation results verify the effectiveness of the CCCP and LBA methods in power allocation and rate control and the MCUG method in user grouping when compared to the greedy and random methods. Besides, the superiority of RSMA for URLLC services is demonstrated when compared to spatial division multiple access.

Supporting ultra-reliable and low-latency communications (URLLC) is crucial for vehicular traffic safety and other mission-critical applications. In this paper, a novel proximity and quality-of-service-aware resource allocation framework for vehicle-to-vehicle (V2V) communication is proposed. The proposed scheme incorporates the physical proximity and traffic demands of vehicles to minimize the total transmission power over the allocated resource blocks (RBs) under reliability and queuing latency constraints. A Lyapunov framework is used to decompose the power minimization problem into two interrelated sub-problems: RB allocation and power optimization. To minimize the overhead introduced by frequent information exchange between the vehicles and the roadside unit (RSU), the resource allocation problem is solved in a semi-distributed fashion. First, a novel RSU-assisted virtual clustering mechanism is proposed to group vehicles into disjoint zones based on mutual interference. Second, a per-zone matching game is proposed to allocate RBs to each vehicle user equipment (VUE) based on vehicles’ traffic demands and their latency and reliability requirements. In the formulated one-to-many matching game, VUE pairs and RBs rank one another using preference relations that capture both the queue dynamics and interference. To solve this game, a semi-decentralized algorithm is proposed using which the VUEs and RBs can reach a stable matching. Finally, a latency-and reliability-aware power allocation solution is proposed for each VUE pair over the assigned subset of RBs. Simulation results for a Manhattan model show that the proposed scheme outperforms the state-of-art baseline and reaches up to 45% reduction in the queuing latency and 94% improvement in reliability.

In real-time wireless control systems, ultra-reliable and low-latency communication (URLLC) is critical for the connection between the remote controller and its control objective. Since both transmission delay and packet loss can lead to control performance loss, our goal is to optimize control performance by jointly considering control and URLLC constraints in this paper. To achieve this goal, we formulate an optimal problem to minimize control cost by optimizing the packet drop and wireless resource allocation. To solve the problem, we analyze the relationship between communication and control. Then, based on the relationship, we decompose the original problem into two subproblems: 1) an optimal packet-drop problem to minimize control cost and 2) an optimal resource allocation problem to minimize communication packet error. Finally, the corresponding solutions for each subproblem can be obtained. Compared with the traditional method only considering the communication aspect, the proposed packet-drop and resource allocation method shows remarkable performance gain in terms of control cost.

Consumer electronics can support sixth-generation (6G) systems and their services, including ultra-reliable and low-latency communications (URLLC). In this context, unmanned aerial vehicles (UAVs) have become increasingly popular because they can: be dynamically positioned, take advantage of channel gains, and communicate directly via line-of-sight. UAVs, on the other hand, are unable to maintain a stable flight for prolonged periods of time and suffer from jittering impairments caused by strong winds. As a result of atmospheric conditions and environmental interference, the perfect channel state information (CSI) becomes obsolete. The aim of this study is to propose a power-efficient resource allocation scheme for URLLC-enabled UAV communication systems under finite block lengths, imperfect CSIs, and adverse jittering effects caused by wind. This involves optimizing UAV positioning and blocklength distribution together. Additionally, we propose a perturbation-based semidefinite programming (SDP) approach to reduce the sum power and demonstrate that it outperforms fixed benchmark algorithms. As such, it can reach power savings up to 77.18% compared to fixed benchmark algorithms. Our extensive simulation results demonstrate that our approach performs similarly to the exhaustive search and has low complexity. Thus, the proposed method thus provides a practical power-efficient URLLC implementation for memory-constrained UAV networks.

Due to the high altitude and deployment flexibility, unmanned aerial vehicles (UAVs) can be used as a relay to avoid obstacles and extend the coverage of wireless networks. On the other hand, ultra-reliable and low-latency communication (URLLC) is often required to deliver the information reliably and timely for many emerging applications. In this correspondence, we combine the advantages of both UAV and URLLC to investigate the resource allocation for a URLLC-enabled two-way UAV relaying system. Our goal is to maximize the transmission rate of the backward link with the constraint of URLLC requirement for the forward link. The optimization is non-convex and difficult to solve. Therefore, the optimization variables are divided to several blocks, and three sub-problems are formulated and solved. Finally, an iterative algorithm is proposed to solve these sub-problems alternately. Simulation results show that the proposed joint optimization scheme can achieve excellent performance for the URLLC-enabled two-way UAV relaying system.

Cloud computing is one among the emerging platforms in business, IT enterprise and mobile computing applications. Resources like Software, CPU, Memory and I/O devices etc. are utilized and charged as per the usage, instead of buying it. A Proper and efficient resource allocation in this dynamic cloud environment becomes the challenging task due to drastic increment in cloud usage. Various promising technologies have been developed to improve the efficiency of resource allocation process. But still there is some incompetency in terms of task scheduling and power consumption, when the system gets overloaded. So an energy efficient task scheduling algorithm is required to improve the efficiency of resource allocation process. In this paper an improved task scheduling and an optimal power minimization approach is proposed for efficient dynamic resource allocation process. Using prediction mechanism and dynamic resource table updating algorithm, efficiency of resource allocation in terms of task completion and response time is achieved. This framework brings an efficient result in terms of power reduction since it reduces the power consumption in data centers. The proposed approach gives accurate values for updating resource table. An efficient resource allocation is achieved by an improved task scheduling technique and reduced power consumption approach. The Simulation result gives 8% better results when comparing to other existing methods.

Hybrid Transactional-Analytical Processing (HTAP) databases face significant challenges in CPU resource allocation due to the conflicting requirements of Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) workloads. Traditional static scheduling approaches fail to adapt to dynamic workload patterns, leading to suboptimal performance and resource utilization inefficiencies. The diverse characteristics of transactional and analytical queries require sophisticated scheduling strategies that can balance latency-sensitive transaction processing with throughput-oriented analytical operations.This study proposes a Hierarchical Deep Reinforcement Learning (HDRL) framework for adaptive CPU scheduling in HTAP database systems. The framework employs a two-level architecture where a high-level agent manages workload prioritization between OLTP and OLAP components, while low-level agents optimize resource allocation within each processing type. Deep Q-Networks (DQN) and Actor-Critic algorithms enable dynamic adaptation to changing workload patterns and system conditions.Experimental evaluation using industry-standard benchmarks demonstrates that the proposed framework achieves 34% improvement in overall system throughput while reducing OLTP query latency by 28% compared to traditional scheduling methods. The hierarchical approach successfully balances competing workload demands and adapts to varying system conditions, resulting in enhanced resource utilization efficiency and improved Quality of Service (QoS) guarantees across both transactional and analytical processing requirements.

Computation offloading via device-to-device communications can improve the performance of mobile edge computing by exploiting the computing resources of user devices. However, most proposed optimization-based computation offloading schemes lack self-adaptive abilities in dynamic environments due to time-varying wireless environment, continuous-discrete mixed actions, and coordination among devices. The conventional reinforcement learning based approaches are not effective for solving an optimal sequential decision problem with continuous-discrete mixed actions. In this paper, we propose a hierarchical deep reinforcement learning (HDRL) framework to solve the joint computation offloading and resource allocation problem. The proposed HDRL framework has a hierarchical actor-critic architecture with a meta critic, multiple basic critics and actors. Specifically, a combination of deep Q-network (DQN) and deep deterministic policy gradient (DDPG) is exploited to cope with the continuous-discrete mixed action spaces. Furthermore, to handle the coordination among devices, the meta critic acts as a DQN to output the joint discrete action of all devices and each basic critic acts as the critic part of DDPG to evaluate the output of the corresponding actor. Simulation results show that the proposed HDRL algorithm can significantly reduce the task computation latency compared with baseline offloading schemes.

Increasing the efficiency of resource allocation is the basis and guarantee for boosting high-quality economic development. Based on the panel data of Chinese industrial enterprises and cities from 2008 to 2013, this paper studies the influence of infrastructure construction demand shocks represented by local government debt expansion on the efficiency of sectoral resource allocation from the perspective of sectoral linkage. According to the empirical findings, local government debt significantly reduces the resource allocation efficiency of manufacturing sector that is highly related to infrastructure construction. This conclusion is still tenable after the robustness test using the simulated local government debt as an instrumental variable. Further mechanism tests show that there are two reasons for the decline of the efficiency of resource allocation in manufacturing sector that is highly related to infrastructure construction. First, more product demands and investments brought by the expansion of local government debt flow to less productive enterprises in the sector. Second, resource misallocation reduces the probability of high-productivity enterprises entering the market and low-productivity enterprises exiting the market, and the effect is more prominent in cities with high dependence on state-owned enterprises and high pressure on officials to be promoted. According to this study, the performance management of local government debt should be further strengthened, and particular attention should be paid to the influence of local government debt on enterprise investment and financing crowding out and resource misallocation.

With the emergence of ultra-reliable and low latency communication (URLLC) services, link adaptation (LA) plays a pivotal role in improving the robustness and reliability of communication networks via appropriate modulation and coding schemes (MCS). LA-based resource management schemes in both physical and medium access control layers can significantly enhance the system performance in terms of throughput, latency, reliability, and quality of service. Increasing the number of retransmissions will achieve higher reliability and increase transmission latency. In order to balance this trade-off with improved link performance for URLLC services, we study a joint subcarrier and power allocation problem to maximize the achievable sum-rate under an appropriate MCS. The formulated problem is mixed-integer nonconvex programming which is challenging to solve optimally. In addition, a direct application of standard optimization techniques is no longer applicable due to the complication of the effective signal-to-noise ratio (SNR) function. To overcome this challenge, we first relax the binary variables to continuous ones and introduce additional variables to convert the relaxed problem into a more tractable form. By leveraging the successive convex approximation method, we develop a low-complexity iterative algorithm that guarantees to achieve at least a locally optimal solution. Simulation results are provided to show the fast convergence of the proposed iterative algorithm and demonstrate the significant performance improvement in terms of the achievable sum-rate, compared with the conventional LA approach and existing retransmission policy.