Reinforcement Learning-Based Resource Partitioning for Improving Responsiveness in Cloud Gaming

In this paper, we study the problem of how to dispatch the play requests to the cloud servers in a cloud gaming system. We show that the dispatching strategy of play requests may heavily affect the total service cost of the cloud gaming system. The play request dispatching problem can be considered as a variant of the dynamic bin packing problem. However, we show that the classical bin packing algorithms such as First Fit and Best Fit are not efficient in dispatching the play requests in cloud gaming due to the diurnal workload pattern of online games. To address this issue, we propose an efficient request dispatching algorithm that assigns the play requests according to the predicted ending times of the game sessions. Through extensive evaluations using real online gaming traces, we show that the proposed dispatching algorithm can significantly reduce the resource waste of the cloud servers and thus decrease the total service cost compared to the First Fit and Best Fit algorithms.

Cloud gaming is promising to provide high-quality game services by outsourcing game execution to cloud so that users can access games via thin clients (e.g., smartphones or tablets). However, existing cloud gaming systems su er from low GPU utilization in the virtualized environment. Moreover, GPU resources are scheduled in units of virtual machines (VMs) and this kind of coarse-grained scheduling at the VM-level fails to fully exploit GPU processing capacity. In this paper, we present ShareRender, a cloud gaming sys- tem that o oads graphics workloads within VMs directly to GPUs, bypassing GPU virtualization. For each game running in a VM, ShareRender starts a graphics wrapper to intercept frame rendering requests and assign them to render agents responsible for frame rendering on GPUs. Thanks to the exible workload assignment among multiple render agents, ShareRender enables ne-grained resource sharing at the frame-level to signi cantly improve GPU utilization. Further more, we design an online algorithm to determine workload assignment and migration of render agents, which considers the tradeo between minimizing the number of active server and low agent migration cost. We conduct experiments on real deployment and trace-driven simulations to evaluate the performance of ShareRender under di erent system settings. The results show that ShareRender outperforms the existing video-streaming-based cloud gaming system by over 4 times.

Massively Multiplayer Online Cloud Game (MMOCG) is a combination of cloud gaming and Massively Multiplayer Online Game (MMOG). Running MMOGs on the cloud can reduce the cost for the players, attract potential players, and meet the nowadays preference of mobile devices. However, the MMOCGs introduce new challenges that need to be addressed. MMOGs usually consume more computational resources because of their sophisticated graphics. Hosting MMOCGs can be more expensive for game operators than hosting other genres of games on the cloud. In order to earn a decent profit, the cost of hosting MMOCGs must be reduced to an acceptable cost. Moreover, MMOCG introduce extra cloud servers to MMOG, which increased the burden of network transmission. So, the MMOCGs must have a good quality of service (QoS) and quality of experience (QoE) to retain the players. In this thesis, we study the server allocation problem to improve MMOCGs in many aspects, including money cost, QoS, and QoE. We proposed three different server allocation strategies to address the challenges of MMOCGs in different situations. We first consider the MMOCG system with several cloud gaming platforms that lie in different locations in the world, each can provide a type of server that can host MMOG on them. The cloud gaming system only obtains the information necessary for gaming. With only the log-in information, a server allocation problem is defined to reduce the money cost of hosting MMOCGs and provide an acceptable network latency for the players. The problem is formulated into an optimization problem that aims to minimize the weighted combination of server rental cost, data transfer cost, and network latency. A genetic algorithm is developed to solve the minimization problem. Extensive experiments are conducted to evaluate the proposed method and compare it with the state-of-the-art as well. The experimental results show that the method gives a lower money cost and a shorter network latency most of the time. We then consider the MMOCG system that can collect the past logging in and out data of the players. We proposed a server allocation problem that aims to reduce the cost of cloud server rental and improve the QoE of the system. A genetic algorithm with gaming time prediction is proposed to solve the problem. The gaming time of the players logging into the system is predicted by the model trained by a neural network. We conduct experiments to evaluate our method and the proposed gaming time prediction model. The results show that our prediction model can achieve almost highly accurate prediction and the server allocation strategy we proposed outperforms the prior art in both server rental and QoE. At last, we consider the influence of QoE on the movement of players during gaming time and propose a server allocation problem aiming to improve the QoE of players during gaming time. We propose a genetic algorithm with trace prediction to address this problem. A trace prediction that predicts the future positions of players for a few time steps when they log into the MMOCG system. Experiments are conducted to evaluate our method by measuring the network latency, mean opinion score, and the time consuming computation. The results show that our method outperforms the genetic algorithm without trace prediction in over 50% of the cases.

Reducing response delay in multivariate process monitoring by a stacked long-short term memory network and real-time contrasts

Cloud gaming has been very popular recently, but providing satisfactory gaming experiences to players at a modest cost is still challenging. Colocating several games onto one server could improve server utilization. To enable efficient colocations while providing Quality of Service (QoS) guarantees, a precise quantification of performance interference among colocated games is required. However, achieving such precise interference prediction is very challenging for games due to the complexity introduced by the contention on many shared resources across CPU and GPU. Moreover, the distinctive properties of cloud gaming require that the prediction model should be constructed beforehand and the prediction should be made instantaneously at request arrivals, which further increases the difficulty. The existing solutions are either not applicable or not effective due to many limitations. In this paper, we present GAugur, a novel methodology that enables highly accurate prediction of the performance interference among games arbitrarily colocated. By leveraging machine learning technologies, GAugur is able to capture the complex relationship between the interference and the contention features of colocated games. We evaluate GAugur through extensive experiments using a large number of real popular games. The results show that GAugur is able to identify whether a colocated game satisfies QoS requirement within an average error of 5%, and is able to quantify the performance degradation of a colocated game within an average error of 7.9%, which significantly outperforms the alternatives. Moreover, GAugur incurs an offline profiling cost linear to the number of games, and negligible overhead for online prediction. We apply GAugur to guiding efficient game colocations for cloud gaming. Experimental results show that GAugur is able to increase the resource utilization by 20% to 60%, and improve the overall performance by up to 15%, compared to the state-of-the-art solutions.

Contrary to conventional gaming-on-demand services that stream gaming video from cloud to players' terminals, a decomposed cloud gaming platform supports flexible migrations of gaming components between the cloud server and the players' terminals. In this paper, we present the design and implementation of the proposed decomposed gaming system. The cognitive resource optimization of the system under distinct targets, including the minimization of cloud, network, and terminal resources and response delay, subject to quality of service (QoS) assurance, is formulated as a graph partitioning problem that is solved by exhaustive searches. Simulations and experimental results demonstrate the feasibility of cognitive resource management in a cloud gaming system to efficiently adapt to variations in the service environments, such as increasing the number of supported devices and reducing the network bandwidth consumption of user terminals, while satisfying different QoS requirements for gaming sessions. We also suggest two heuristic algorithms based on local greedy and genetic algorithm approaches, which can potentially provide scalable but suboptimal solutions in large-scale implementations.

With the rapid development of cloud computing technologies, cloud gaming has gained wide attention from the industrial field. By executing the gaming software on the cloud and transiting the rendered game scenes back to the client, game players can play games anywhere on any device. However, the centralized architecture of existing cloud gaming platforms causes serious network latency, which deteriorates user experience significantly. To reduce network latency and relieve the workload on the mobile client, we propose and implement a cloudlet-based mobile cloud gaming system design. By deploying the cloudlets in the nearby region of mobile users, we can provide computational resources to the mobile clients and complete tasks such as gaming state maintenance, game scene rendering, etc. The system can also reduce the gaming latency and the workload on the client greatly. We use the open-source cloud computing software to build a cloudlet system and develop an Android-based cloud gaming client, which can receive and display gaming video streams, and map and upload the game operations. Finally, we conduct experimental evaluation of our mobile cloud gaming prototype. The results show that our proposed solution can significantly reduce the fraction of network latency.

The arterial baroreflex plays important role in cardiovascular regulations during exercise. Previous studies showed that the spontaneous sensitivity of the cardiac component of arterial baroreflex (SBRS) is progressively attenuated with increasing workloads of dynamic exercise. However, dynamic modulation of SBRS to changes in workloads is unknown. PURPOSE: We aimed to investigate the amplitude response and phase response of SBRS during cycling exercise with sinusoidally-varying workload. METHODS: We studied 8 healthy male volunteers. After 5 minutes of resting measurements, the subjects performed cycling exercise at 50 W for 5 minutes. Then the workload was changed in a sinusoidal pattern between 30 W to 70 W for 32 minutes. The periods of sinusoidal workload were set to 4, 8 and 16 minutes. Subjects performed each of those three sinusoidal load exercise on separate days in random order. SBRS was evaluated as the slopes of the linear relations between R-R interval and systolic arterial blood pressure (SAP) during spontaneous sequences of at least three consecutive beats when SAP and R-R interval changed in the same directions. SBRS for all sequences were superimposed every periods of sinusoidal load and fitted to a sine model using the least-squares error approach to calculate amplitude response and phase response. RESULTS: During sinusoidal exercise, SBRS changed sinusoidally with the change of workload. The amplitude response of SBRS for 4, 8 and 16 minutes periods were 1.4 ± 0.3, 1.4 ± 0.2, and 1.2 ± 0.3 msec/mmHg, respectively and were not significantly different (p > 0.05). The phase lag of the SBRS from the sinusoidally-varying workload for 4, 8 and 16 minutes periods were 20.4 ± 4.2, 17.4 ± 4.3, and 3.6 ± 4.2 degrees, respectively. The phase lag of the SBRS in 16 minutes period was significantly smaller compared to 4 minutes period (p < 0.05). CONCLUSIONS: We conclude that the arterial baroreflex control of heart periods is modulated continuously in response to changes in workload. When workload changes fast, the modulation of SBRS delays from the changes in workloads. However, SBRS is modulated with almost no delay in response to relatively slow changes in workloads. We suggest that the dynamic modulation of the SBRS is one of the mechanisms to regulate cardiovascular responses in accordance with changes in workloads.

*† The main elements in this project were to observe the change in operator workload and performance with the increase of transport delay in the visual system, the level of simulator sickness and attempt to determine if one can use performance and workload measurements to evaluate the level of sickness in this particular simulator. Experiments were conducted at the Man-Machine Systems Laboratory at the SUNY at Binghamton. The simulator facility utilized during the experiments was the Advanced Mobile Operation Simulator (AMOS II), manufactured by the DORON Precision Corp. Four different added visual transport delay conditions were subject to investigation: 0 ms, 33 ms, 99 ms, 165 ms. Various performance parameters, such as steering wheel deflection, accelerator and break pedal usage, vehicle state, etc., were measured during each experimental run. However, for this particular study the performance analysis was concentrated on the steering activity data. The Power Spectrum Density of the steering control input was evaluated and statistically analyzed. The thrust of the paper is the comparison between the expected results (changes in workload level, performance changes and simulator sickness level due to visual delay in the system) and the collected data. Results are presented and conclusions are drawn.

Cloud gaming systems host the game in the cloud, and stream players' gaming videos to the terminals in the form of encode video frames. To address the high bandwidth issue of real-time gaming video transmission, a cloudlet-assisted multiplayer cloud gaming system was proposed to encourage cooperative video sharing via a secondary ad-hoc network, on the purpose of exploiting the similarities of video frames among multiple players in a same game. However, the video cooperative sharing among players also introduces fairness problems. In this paper, we complete the ad hoc cloudlet-assisted cloud gaming system by further considering the mobility of terminal devices and the diversity of network quality for distinct players. With mathematical formulation, we study the players' behavior in cooperative sharing patterns and propose a reputation-based multiplayer fairness scheme in terms of frame encoding. Experimental results illustrate the impact of mobility on the system performance and evaluate that the proposed solution provides better fairness gaming ecosystem compared to the existing platform.

Good user experience with interactive cloud-based multimedia applications, such as cloud gaming and cloud-based VR, requires low end-to-end latency and large amounts of downstream network bandwidth at the same time. In this paper, we present a foveated video streaming system for cloud gaming. The system adapts video stream quality by adjusting the encoding parameters on the fly to match the player's gaze position. We conduct measurements with a prototype that we developed for a cloud gaming system in conjunction with eye tracker hardware. Evaluation results suggest that such foveated streaming can reduce bandwidth requirements by even more than 50% depending on parametrization of the foveated video coding and that it is feasible from the latency perspective.

BackgroundIndividual bees exhibit complex movement patterns to efficiently exploit small areas within larger plant populations. How such individual spatial behaviours scale up to the collective level, when several foragers visit a common area, has remained challenging to investigate, both because of the low resolution of field movement data and the limited power of the statistical descriptors to analyse them. To tackle these issues we video recorded all flower visits (N = 6205), and every interaction on flowers (N = 628), involving foragers from a bumblebee (Bombus terrestris) colony in a large outdoor flight cage (880 m2), containing ten artificial flowers, collected on five consecutive days, and analysed bee movements using networks statistics.ResultsBee-flower visitation networks were significantly more modular than expected by chance, indicating that foragers minimized overlaps in their patterns of flower visits. Resource partitioning emerged from differences in foraging experience among bees, and from outcomes of their interactions on flowers. Less experienced foragers showed lower activity and were more faithful to some flowers, whereas more experienced foragers explored the flower array more extensively. Furthermore, bees avoided returning to flowers from which they had recently been displaced by a nestmate, suggesting that bees integrate memories of past interactions into their foraging decisions.ConclusionOur observations, under high levels of competition in a flight cage, suggest that the continuous turnover of foragers observed in colonies can led to efficient resource partitioning among bees in natural conditions.

Cloud gaming allows users to play high-quality games on low-end devices by offloading game processing to the cloud. However, network latency remains a significant issue affecting the gaming experience. Speculative execution is a promising approach to hide network latency by predicting and transmitting future frames early. However, existing methods generate excessive compute load and network traffic due to many potential input patterns. This paper introduces a pattern reduction method that uses a bit field representation of the input and facilitates efficient speculative execution in cloud games. There are two pattern reduction techniques: analyzing temporal patterns to detect frequent transitions and using LSTM-based predictions to estimate input probabilities. Experiments using actual gaming data show that the proposed methods significantly reduce rendered frames and network traffic versus prior speculative execution methods. The results demonstrate the method's effectiveness and scalability across diverse game genres.

The combination of cloud computing and advancements in GPU have made many real time services possible, including Cloud Gaming (CG). Doing all the process-intensive tasks in the cloud frees players from upgrading their heterogeneous devices and installing new software, and lets them play wherever and whenever. However, higher quality is always demanded by players. For instance, as frame rate has major impact on the player’s gaming performance, demand of higher frame rate is increasing. On the other hand, service providers aim to offer cost effective services. Management of the graphic-intensive CG service demand and maximizing the service providers' benefits is an issue that must be addressed properly. As remote GPU plays the main role of rendering and is the most expensive infrastructure rented by the service provider, its appropriate management is vital to address the above issue. To do so, we formulate the problem in an efficient manner and propose two methods to maximize both GPU utilization and the users’ quality of experience (QoE) at the same time, subject to the constraints of the servers. Our methods are based on two metaheuristic algorithms to solve an NP-Hard optimization problem for GPU-based server selection. Our simulation results shows that by increasing the number of players, both algorithms have increasing performance in terms of GPU utilization, reduced capacity wastage, and QoE.

While cloud-edge infrastructures demand flexible and sophisticated resource management, 6G networks necessitate very low latency, great dependability, and broad connection. Cloud computing’s scalability and agility enable it to prioritize service delivery at various levels of detail while serving billions of users. However, due to resource inefficiencies, virtual machine (VM) issues, response delays, and deadline violations, real-time task scheduling is challenging in these settings. This study develops an AI-powered task scheduling system based on the newly published Unfair Semi-Greedy (USG) algorithm, Earliest Deadline First (EDF), and Enhanced Deadline Zero-Laxity (EDZL) algorithm. The system chooses the best scheduler based on load and work criticality by combining reinforcement learning adaptive logic with a dynamic resource table. Over 10,000 soft real-time task sets were utilized to evaluate the framework across various cloud-edge scenarios. When compared to solo EDF and EDZL solutions, the recommended hybrid method reduced average response times by up to 26.3% and deadline exceptions by 41.7%. The USG component achieved 98.6% task stimulability under saturated edge settings, indicating significant changes in workload. These findings suggest that the method might be useful for applications that need a speedy turnaround. This architecture is especially well-suited for autonomous systems, remote healthcare, and immersive media, all of which require low latency and dependability, and it may be extended to AI-native 6G networks.