• Home
  • Search
  • Deep Reinforcement Learning for Online Cloud Resource Management: An Approximate Dynamic Programming Perspective
  • https://doi.org/10.62311/nesx/rp-m4-dec-2021Copy DOI Icon

Deep Reinforcement Learning for Online Cloud Resource Management: An Approximate Dynamic Programming Perspective

  • Abstract
  • Literature Map
  • Similar Papers
Abstract

Abstract: Online resource management in cloud computing must respond to rapidly changing demand and system conditions, often exceeding the adaptability of static or periodically recomputed optimisation plans. This study formulates cloud resource management as a Markov decision process (MDP) and investigates deep reinforcement learning (DRL) as an approximate dynamic programming (ADP) approach for deriving adaptive control policies. The system state encompasses workload, capacity utilisation, and performance indicators, while actions govern virtual machine placement, scaling, and server power-state transitions. Reward functions combine energy and operating costs with penalties associated with service-level agreement violations. DRL algorithms, including deep Q-networks, advantage actor–critic, and proximal policy optimization, are implemented in PyTorch and trained through interaction with a CloudSim-based simulation environment calibrated to realistic load and power profiles. The learned policies are evaluated against deterministic and scenario-based stochastic optimisation baselines using metrics such as average cost, energy consumption, SLA violations, and robustness to non-stationary demand. Results show that DRL policies can approach or surpass the performance of stochastic optimisation solutions in highly dynamic scenarios, while maintaining feasible control actions. The study demonstrates how DRL can be positioned within an OR/ADP framework and highlights conditions under which learning-based methods offer significant advantages for online cloud control. Keywords: deep reinforcement learning, approximate dynamic programming, cloud computing, Markov decision processes, online resource management, energy-aware control, PyTorch, simulation

Similar Papers
  • Conference Article
  • Citations3

Resource Management in Cloud Based on Deep Reinforcement Learning

  • Jul 01, 2022
  • Chuyi Liu +5
  • Research Article
  • Citations54

Energy efficient task scheduling based on deep reinforcement learning in cloud environment: A specialized review

  • Oct 14, 2023
  • Future Generation Computer Systems
  • Huanhuan Hou +2
  • Research Article
  • Citations1

Deep Reinforcement Learning for irrigation optimization: Advantages, opportunities, and challenges

  • Dec 01, 2025
  • Agricultural Water Management
  • Jiamei Liu +8
  • Research Article

Optimizing thermoelectric energy harvesting using deep reinforcement learning for dynamic energy management and system efficiency

  • Dec 13, 2025
  • Scientific Reports
  • Chirayu Nilesh Chaudhari +6
  • PDF
  • Research Article
  • Citations22

A Review of Mobile Robot Path Planning Based on Deep Reinforcement Learning Algorithm

  • Dec 01, 2021
  • Journal of Physics: Conference Series
  • Yanwei Zhao +2
  • PDF
  • Research Article
  • Citations5

Deep imitation reinforcement learning with expert demonstration data

  • Oct 31, 2018
  • The Journal of Engineering
  • Menglong Yi +3
  • Research Article
  • Citations28

Logistics-involved task scheduling in cloud manufacturing with offline deep reinforcement learning

  • May 08, 2023
  • Journal of Industrial Information Integration
  • Xiaohan Wang +3
  • PDF
  • Research Article
  • Citations38

Deep Reinforcement Learning Unleashing the Power of AI in Decision-Making

  • Feb 02, 2024
  • Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023
  • Jeff Shuford
  • Conference Article

Safe Deep Reinforcement Learning Based on Sample Value Evaluation

  • Dec 02, 2022
  • Rongjun Ye +2
  • Research Article
  • Citations1

DDPG Agent to Swing Up and Balance Cart- Pole System

  • Apr 09, 2021
  • International Journal of Advanced Research in Science, Communication and Technology
  • Buvanesh Pandian V
  • Conference Article
  • Citations36

An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents

  • Aug 01, 2019
  • Felipe Petroski Such +10
  • Research Article

Real-Time Deep Reinforcement Learning Policy Controller for Magnetically Levitated Robots

  • Jan 01, 2026
  • IEEE Transactions on Automation Science and Engineering
  • Yang Wang +3
  • Conference Article
  • Citations2

A Motion Planning and Control Method of Quadruped Robot Based on Deep Reinforcement Learning

  • Dec 05, 2022
  • Weilong Liu +5
  • Research Article
  • Citations31

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

  • Jun 01, 2023
  • IEEE Communications Magazine
  • Peng Cheng +5
  • Research Article
  • Citations97

Intelligent User Association for Symbiotic Radio Networks Using Deep Reinforcement Learning

  • Jul 01, 2020
  • IEEE Transactions on Wireless Communications
  • Qianqian Zhang +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.