Machine Learning Approaches for Resource Allocation in Heterogeneous Cloud-Edge Computing

Health care includes automatic analysis and diagnosis with the help of various learning approaches, like machine learning, deep learning, natural language processing (NLP), etc. Nowadays, machine learning plays a vital role in the field of health care analysis, with its simplified approach. In addition, machine learning algorithms represent the state of the art in the field of diagnosis. Therefore, machine learning approaches to analysis in the field of healthcare informatics will help in analysis and early diagnosis, which will reduce the processing time. Machine learning strategies improve the medical image processing, especially in healthcare informatics. Up until the past decade, it was a challenging task to achieve accurate results in healthcare informatics, but machine learning has changed the scenario and made it easier. Healthcare is one the rapidly growing areas, where technology plays a vital role in the end-to-end process, especially in analysis and diagnosis. These strategies give better results and are user friendly. This chapter will describe the different machine learning modules used in the field of health care, with informatics based on medical image processing.

It has been a long-held belief that judicious resource allocation is critical to mitigating interference, improving network efficiency, and ultimately optimizing wireless communication performance. The traditional wisdom is to explicitly formulate resource allocation as an optimization problem and then exploit mathematical programming to solve the problem to a certain level of optimality. Nonetheless, as wireless networks become increasingly diverse and complex, e.g., in the high-mobility vehicular networks, the current design methodologies face significant challenges and thus call for rethinking of the traditional design philosophy. Meanwhile, deep learning, with many success stories in various disciplines, represents a promising alternative due to its remarkable power to leverage data for problem solving. In this paper, we discuss the key motivations and roadblocks of using deep learning for wireless resource allocation with application to vehicular networks. We review major recent studies that mobilize the deep learning philosophy in wireless resource allocation and achieve impressive results. We first discuss deep learning assisted optimization for resource allocation. We then highlight the deep reinforcement learning approach to address resource allocation problems that are difficult to handle in the traditional optimization framework. We also identify some research directions that deserve further investigation.

The growing number of complex and heterogeneous Internet of Things (IoT) applications has imposed a high demand for scarce communications and computing resources. To meet this stringent requirement, it is desirable to develop large-scale highly adaptive online resource allocation strategies to streamline existing network operations. Deep reinforcement learning (DRL), which combines the merits of reinforcement learning and deep learning, is capable of addressing complex decision-making tasks, thus enabling efficient online resource allocation. In this article, we present a DRL-based resource allocation framework. We begin a discussion on DRL basics and review its several recent applications. Then, we develop two new DRL algorithms that facilitate unlocking the potential of DRL and offer viable solutions to many more complex resource allocation problems. The first one tackles an optimization problem exposed to mixed (discrete and continuous) action spaces and bound by a number of highly non-linear quality-of-service (QoS) constraints. The second one extends the single-agent DRL to a more challenging multi-agent DRL by introducing a novel semi-distributed architecture. Finally, we discuss the challenges and future visions of applying DRL to real-world IoT networks.

Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradation within an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner.

With the widespread application of cloud computing technology, the energy consumption problem of data centers is becoming increasingly severe. This research is dedicated to presenting a highly efficient and smart solution for optimizing energy usage in cloud computing systems, especially since effectively managing energy consumption through resource allocation and task scheduling has now become a critical challenge in the field. In view of this, the study first constructs a resource allocation model based on redundant resource recovery to dynamically identify and recover redundant resources. Second, a task scheduling optimization method based on dual deep reinforcement learning is established, which utilizes directed acyclic graphs to model task dependencies and optimizes task execution order and resource allocation strategies through reinforcement learning agents. In the testing of the resource allocation optimization model, when the number of tasks is 500, the redundant resource recovery rate is 87.0%, and the computational complexity is 16.7 s. The response time in the regional resource environment is 2.28 s, and the load adaptability is 95.8%. In the task scheduling method test, the proposed method has a memory occupancy of 85 MB and a scheduling delay of 0.13 s in a high-bandwidth environment of 300 Mbps. The results show that the proposed method outperforms traditional scheduling strategies in terms of resource utilization and energy consumption, and contributes to achieving the goal of green cloud computing. The research aims to provide an efficient and intelligent solution for energy optimization in cloud computing systems.

Recently, deep neural networks (DNNs) have been widely studied and achieved tremendous success in a variety of real-world applications, such as computer vision, natural language processing, medical diagnosis, and machine translation. Deep reinforcement learning (DRL), as an emerging powerful deep learning technique, combines DNNs with reinforcement learning into an interactive system that enables the agent to learn the best policy from the environment to optimize their rewards. DRL opens up many new applications in domains such as healthcare, robotics, and smart grids. With the rapid evolution of IT infrastructures, cloud computing has been witnessed as the most common computing paradigm. The underlying infrastructure of cloud computing relies on a large amount of data centers. Nowadays, with the fast development of communication technologies, such as 5G, cloud computing is now evolving into yet another new paradigm called edge computing, providing for emerging applications, especially in the Internet-of-Things. The energy efficiency issue from both "cloud" and "edge" are becoming more crucial and calls for more attention. To solve the real-world energy efficiency problems, we take advantage of the deep learning and deep reinforcement learning techniques for efficient modeling of both "edge" and "cloud" applications. On the "edge" side, wearable technologies and the Internet of Things (IoT) are becoming increasingly popular in both personal and commercial applications with the employment of the embedded system technologies. Non-Volatile Processors (NVPs) have been proposed for the intermittent power supply problem, to ensure the instant on/off for embedded systems under the unstable power supply. We proposed neural network-based prediction techniques to derive the best power extraction policy and converter parameters optimization technique for NVPs. In cloud computing, we formulated a novel optimal power management problem of data centers to minimize the total cost by proposing the long short term memory (LSTM) neural network-based method for the prediction of future data center power consumption, and further we presented a novel DRL-based hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. Furthermore, we explored the effectiveness of the DRL-based framework for high dimensional action and state spaces problem-dynamic treatment regimes, to model real-life complexity in heterogeneous disease progression and treatment choices, to provide doctors and patients the data-driven personalized decision recommendations. Last but not least, we explored the efficiency of deep neural networks. Structured weight pruning is a representative model compression technique of DNNs to reduce both the storage and computation requirements and meanwhile accelerate inference. An automatic hyperparameter determination process is necessary due to a large number of flexible hyperparameters. We propose an automatic structured pruning framework to compress the DNNs. Our framework outperforms the prior works on automatic model compression by up to 33× in pruning rate (120× reduction in the actual parameter count) under the same accuracy. Significant inference speedup has been observed from the proposed framework on actual measurements on the smartphone.

The proliferation of multi-cloud environments has introduced complexities in resource management, necessitating intelligent solutions for optimal resource allocation. Traditional resource allocation techniques often struggle to adapt to the dynamic nature of multi-cloud ecosystems, leading to suboptimal performance, increased costs, and inefficient resource utilization. This paper explores the application of Deep Reinforcement Learning (DRL) as an advanced AI-driven approach to optimize resource distribution across multiple cloud platforms. By leveraging reinforcement learning techniques, DRL enables autonomous decision-making, learning from past experiences to refine resource allocation strategies in real-time. The proposed framework is designed to handle diverse workloads, minimize latency, and maximize cost-efficiency. Additionally, the study evaluates key performance metrics, including throughput, response time, and adaptability, comparing DRL-based approaches with traditional methods. Experimental results indicate that DRL significantly improves efficiency, scalability, and adaptability in dynamic cloud environments, paving the way for intelligent and automated cloud resource management.

With the rapid evolution of large language models (LLM), reinforcement learning (RL) has emerged as a pivotal technique for code generation and optimization in various domains. This paper presents a systematic survey of the application of RL in code optimization and generation, highlighting its role in enhancing compiler optimization, resource allocation, and the development of frameworks and tools. Subsequent sections first delve into the intricate processes of compiler optimization, where RL algorithms are leveraged to improve efficiency and resource utilization. The discussion then progresses to the function of RL in resource allocation, emphasizing register allocation and system optimization. We also explore the burgeoning role of frameworks and tools in code generation, examining how RL can be integrated to bolster their capabilities. This survey aims to serve as a comprehensive resource for researchers and practitioners interested in harnessing the power of RL to advance code generation and optimization techniques.

Low-Earth orbit (LEO) satellites have limited on-board resources, user terminals are unevenly distributed in the constantly changing coverage area, and the service requirements vary significantly. It is urgent to optimize resource allocation under the constraint of limited satellite spectrum resources and ensure the fairness of service admission control. Therefore, we propose an intelligent hierarchical admission control (IHAC) strategy based on deep reinforcement learning (DRL). This strategy combines the deep deterministic policy gradient (DDPG) and the deep Q network (DQN) intelligent algorithm to construct upper and lower hierarchical resource allocation and admission control frameworks. The upper controller considers the state features of each ground zone and satellite resources from a global perspective, and determines the beam resource allocation ratio of each ground zone. The lower controller formulates the admission control policy based on the decision of the upper controller and the detailed information of the users' services. At the same time, a designed reward and punishment mechanism is used to optimize the decisions of the upper and lower controllers. The fairness of users' services admissions in each ground zone is achieved as far as possible while ensuring the reasonable allocation of beam resources among zones. Finally, online decision-making and offline learning were combined, so that the controller could make full use of a large number of historical data to learn and generate intelligent strategies with stronger adaptive ability while interacting with the network environment in real time. A large number of simulation results show that IHAC has better performance in terms of a successful service admission rate, service drop rate, and fair resource allocation. Among them, the number of accepted services increased by 20.36% on average, the packet loss rate decreased by 17.56% on average, and the resource fairness increased by 17.16% on average.

This conference note describes a Deep Reinforcement Learning architecture specifically designed to improve wireless network performance over a heterogeneous airborne wireless network consisting of multiple waveforms, antennas, platforms, link protocols, frequencies, spatial transmission, and codes. The cooperative optimization of this high dimensional space is a difficult problem which obviously has a highly correlated characterization where human network operators cannot possibly capture these correlations. Model-free Reinforcement Learning techniques represent a potential solution to our problem. Specifically, we use Deep Q-Learning Networks (DQN) to improve networking performance. We have developed a high-fidelity network simulation tool we call Tactical Airborne Network Simulator (TANS) which we use to train our neural network before deploying to the field where the asset is deployed to some mission which is hopefully somewhat similar to the scenarios used for training. By utilizing the model developed under the TANS training scenarios for the target mission scenario our learning technique gets a head start, rather than using a truly model-free approach. Our technique is codified in the ML community as "Deep Transfer Learning" [4] where terms and metrics have been examined. This paper represents an initial investigation into both the decision support agent architecture and the ML technique. Upcoming research will be described below including our vision for an expanded agent architecture as well as ideas for improved ML techniques which ultimately will result in better wireless network performance. Here we demonstrate a minor throughput performance improvement of 4% using a proof of concept agent over the use of a standard unassisted network. We improved the throughput from 309kbps to 324 kbps.

In India's economy, agriculture is crucially significant. Agriculture automation is a major source of concern and a hot topic all around the world. The world's population is constantly growing, and with it comes increased demand for food as well as occupation. The farmers' old methods were inadequate to satisfy these requirements. As a result, new automated approaches were proposed. These new methods met food demands while simultaneously providing work opportunities for billions of people. Several methods are used for efficient farm management like IoT, cloud computing, AI, machine learning, deep learning, big data, etc. Among these big data is an emerging research area for crop yield prediction. Traditionally, these forecasts were reliant on the farmers' expertise, but now they can roughly anticipate the crop output on their farm by employing a variety of methods. In this paper, we have proposed a recommendation model along with an algorithm to evaluate the future year's crop production. We have compared the random forest machine learning algorithm with the big data algorithm. A brief comparison has been made with these algorithms. The proposed model has been implemented using python.

Explore the article titled Performance Analysis of Machine Learning Approaches in Heart Disease Prediction from IJIRT Volume 12, Issue no. This study evaluates the effectiveness of teaching programs on waste management knowledge among women.

Efficient resource scheduling is essential for maximal utilization of expensive deep learning (DL) clusters. Existing cluster schedulers either are agnostic to machine learning (ML) workload characteristics, or use scheduling heuristics based on operators' understanding of particular ML framework and workload, which are less efficient or not general enough. In this article, we show that DL techniques can be adopted to design a generic and efficient scheduler. Specifically, we propose DL2, a DL-driven scheduler for DL clusters, targeting global training job expedition by dynamically resizing resources allocated to jobs. DL2 advocates a joint supervised learning and reinforcement learning approach: a neural network is warmed up via offline supervised learning based on job traces produced by the existing cluster scheduler; then the neural network is plugged into the live DL cluster, fine-tuned by reinforcement learning carried out throughout the training progress of the DL jobs, and used for deciding job resource allocation in an online fashion. We implement DL2 on Kubernetes and enable dynamic resource scaling in DL jobs on MXNet. Extensive evaluation shows that DL2 outperforms fairness scheduler (i.e., DRF) by 44.1 percent and expert heuristic scheduler (i.e., Optimus) by 17.5 percent in terms of average job completion time.

The rapid advancements in heterogeneous parallel architectures, consisting of CPUs, GPUs, FPGAs, have introduced significant challenges in efficient resource management for high-performance computing systems. Static and heuristic-based approaches fail to address the adaptability required for handling varying workloads and hardware configurations which results in suboptimal performance and energy inefficiency. This research proposes a machine learning-driven adaptive resource management framework that dynamically optimizes task scheduling, resource allocation, and data placement. The framework employs regression models & reinforcement learning algorithms to predict workload behaviors, resource utilization, and task execution times in real time. Experimental results on heterogeneous testbed demonstrate a 21% reduction in task execution time, 18% improvement in energy efficiency, and 38% decrease in fault recovery time compared to conventional methods. These findings highlight the framework’s ability to improve resource utilization while maintaining reliability and minimizing energy overhead. The work advances the field by introducing a unified approach that integrates machine learning for runtime optimization across heterogeneous systems. Practical implications include its applicability to large-scale scientific simulations and deep learning tasks, where adaptive resource management is critical. Future study can focus on enhancing prediction accuracy by advanced deep learning techniques and extending the framework to handle emerging hardware accelerators and edge computing environments.

Considering bandwidth-hungry and low latency requirement of vehicular communications applications, we propose a novel dynamic reinforcement learning-based slicing framework and optimization solutions for efficient resource provisioning in virtualized network for D2D-based vehicle-to-vehicle (V2V) communication. The aim is to balance resource utilization and quality of service (QoS) satisfaction levels for multiple slices. The slicing framework is designed as a three-stage layered framework. In the first stage, we propose dynamic deep reinforcement learning-based virtual resource allocation scheme to allocate distinct resources to slices. In the second stage, we aggregate the D2D resource portion of the slice resource for D2D-based V2V communication. In the third stage, due to the computational complexity and signaling overhead of the physical resource allocation, we transform the problem into a convex optimization problem and solve with an alternating direction method of multipliers-based distributed algorithm. Performance results are provided in terms of resource utilization, QoS satisfaction and throughput to show the benefit of integrating resource slices dedicated to supporting interslice D2D-based V2V communication in vehicular network.