A resource scheduling method for reliable and trusted distributed composite services in cloud environment based on deep reinforcement learning

The global optimal Web service selection based on quality of service (QoS) in cloud environment has become a research focus when there are lots of the same or similar services. In this environment, it is possible that many service users request the same or similar services at the same time, which will result in users' unsatisfied requirement and services' load imbalance. The existing service selection approaches usually suppose that service's load capacity is infinite and user always select the service with the best expected QoS despite the amount of user requests. Therefore, it is very important problem how to get a tradeoff between the users' satisfied requirement and service load balance. To solve this problem, this paper presents a global optimal service selection approach based on QoS and load-aware in cloud environment. In this approach, we first build a user QoS utility model which describes the relationship between user's request and services' QoS, and design a service's load capacity model to achieve the load capacity of a service. Then, we use 0-1 integer programming to build a global optimal model based on QoS utility of users and services' load capacity, and provide the optimal service selection algorithm for users. Finally, by conducting large-scale experiments based on a Web service dataset, we show that our approach can effectively help users to select high qualified services while keeping load balance of services in cloud environment.

Path planning in an unknown environment is a basic task for mobile robots to complete tasks. As a typical deep reinforcement learning, deep Q-network (DQN) algorithm has gained wide popularity in path planning tasks due to its self-learning and adaptability to complex environment. However, most of path planning algorithms based on DQN spend plenty of time for model training and the learned model policy depends only on the information observed by sensors. It will cause poor generalization capability for the new task and time waste for model retraining. Therefore, a new deep reinforcement learning method combining DQN with prior knowledge is proposed to reduce training time and enhance generalization capability. In this method, a fuzzy logic controller is designed to avoid the obstacles and help the robot avoid blind exploration for reducing the training time. A target-driven approach is used to address the lack of generalization, in which the learned policy depends on the fusion of observed information and target information. Extensive experiments show that the proposed algorithm converges faster than DQN algorithm in path planning tasks and the target can be reached without retraining when the path planning task changes.

Trust management is one of the most challenging issues in the emerging cloud computing area. Over the past few years, many studies have proposed different techniques to address trust management issues. However, despite these past efforts, several trust management issues such as identification, privacy, personalization, integration, security, and scalability have been mostly neglected and need to be addressed before cloud computing can be fully embraced. In this article, we present an overview of the cloud service models and we survey the main techniques and research prototypes that efficiently support trust management of services in cloud environments. We present a generic analytical framework that assesses existing trust management research prototypes in cloud computing and relevant areas using a set of assessment criteria. Open research issues for trust management in cloud environments are also discussed.

In this paper, we investigate the user pairing and power allocation scheme for multiple cellular users (CUs) under the downlink non-orthogonal multiple access (NOMA) system. To maximize the sum-rate of all CUs, we formulate the resource allocation issue through optimizing the user pairing relationship and power allocation method. However, due to the nonconvex property of the formulated problem, the original problem is decoupled into two subproblems. First, the optimal power allocation scheme with a given subchannel assignment is obtained via a closed-form solution. Furthermore, based on the obtained optimal allocation scheme, a classical deep reinforcement learning (DRL) method called Deep Q-Network (DQN) algorithm is adopted to find the optimal user pairing scheme, where the DQN algorithm is characterized by higher learning efficiency and better performance of the features extraction ability compared with traditional reinforcement learning (RL) schemes. Simulation results validate the effectiveness of our proposed resource allocation, as compared against the RL based scheme and conventional orthogonal multiple access (OMA) method.

Cloud manufacturing is a new service-oriented manufacturing model that integrates distributed manufacturing resources to provide on-demand manufacturing services over the Internet. Service composition that builds larger-granularity, value-added services by combining a number of smaller-granularity services to satisfy consumers’ complex requirements is an important issue in cloud manufacturing. Meta-heuristic algorithms such as genetic algorithm, particle swarm optimization, and ant colony algorithm are frequently employed for addressing service composition issues in cloud manufacturing. However, these algorithms require complex design flows and lack adaptability to dynamic environment. Deep reinforcement learning provides an alternative approach for solving cloud manufacturing service composition issues. This chapter proposes a deep Q-network (DQN) based approach for service composition in cloud manufacturing, which is able to find optimal service composition solutions through repeated training and learning. Results of experiments that take into account changes of service scales and service unavailability reveal the scalability and robustness of the DQN algorithm-based service composition approach.

Antenna selection is often used for physical layer security to implement secure communications. However, due to the rapid changes of the main channel and the feedback delay of the channel state information (CSI), the transmitter obtains outdated CSI, and the outdated CSI leads to the outdated optimal transmit antenna. In order to improve the security of the system based on outdated CSI, in this paper, we propose a deep reinforcement learning framework of Deep Q Network (DQN) to predict the optimal transmit antenna in the multiple input multiple output (MIMO) wiretap channel. The legitimate receiver receives the pilot signals from each transmitting antenna, and the signal-to-noise ratio (SNR) of the pilot signals transmitted by each transmitting antenna can be obtained through maximal ratio combining. And then the legitimate receiver uses the DQN to predict the transmitting antenna at the next moment according to these SNRs. The simulation results show that DQN algorithm can effectively predict the optimal antenna at the next moment, and reduce the secrecy outage probability of MIMO wiretap system, compared with the traditional algorithm.

Autonomous navigation of mobile robots is a challenging task that requires them to travel from their initial position to their destination without collision in an environment. Reinforcement Learning methods enable a state action function in mobile robots suited to their environment. During trial-and-error interaction with its surroundings, it helps a robot to find an ideal behavior on its own. The Deep Q Network (DQN) algorithm is used in TurtleBot 3 (TB3) to achieve the goal by successfully avoiding the obstacles. But it requires a large number of training iterations. This research mainly focuses on a mobility robot’s best path prediction utilizing DQN and the Artificial Potential Field (APF) algorithms. First, a TB3 Waffle Pi DQN is built and trained to reach the goal. Then the APF shortest path algorithm is incorporated into the DQN algorithm. The proposed planning approach is compared with the standard DQN method in a virtual environment based on the Robot Operation System (ROS). The results from the simulation show that the combination is effective for DQN and APF gives a better optimal path and takes less time when compared to the conventional DQN algorithm. The performance improvement rate of the proposed DQN + APF in comparison with DQN in terms of the number of successful targets is attained by 88%. The performance of the proposed DQN + APF in comparison with DQN in terms of average time is achieved by 0.331 s. The performance of the proposed DQN + APF in comparison with DQN average rewards in which the positive goal is attained by 85% and the negative goal is attained by −90%.

Path planning is a key task in mobile robots, and the application of Deep Q Network (DQN) algorithm for mobile robot path planning has become a hotspot and challenge in current research. In order to solve the obstacle avoidance limitations faced by the DQN algorithm in indoor robot path planning, this paper proposes a solution based on an improved DQN algorithm. In view of the low learning efficiency of the DQN algorithm, the Duel DQN structure is introduced to enhance the performance and combined with a Prioritized Experience Replay (PER) mechanism to ensure the stability of the robot during the learning process. In addition, the idea of Munchausen Deep Q Network (M-DQN) is incorporated to guide the robot to learn the optimal policy more effectively. Based on the above improvements, the PER-D2MQN algorithm is proposed in this paper. In order to validate the effectiveness of the proposed algorithm, we conducted multidimensional simulation comparison experiments of the PER-D2MQN algorithm with DQN, Duel DQN, and the existing methodology PMR-DQN in the Gazebo simulation environment and examined the cumulative and average rewards for reaching the goal point, the number of convergent execution steps, and the time consumed by the robot in reaching the goal point. The simulation results show that the PER-D2MQN algorithm obtains the highest reward in both static and complex environments, exhibits the best convergence, and finds the goal point with the lowest average number of steps and the shortest elapsed time.

Purpose Data service (DS) is a special software service that enables data access in cloud environment and provides a unified data model for cross-origination data integration and data sharing. The purpose of the work is to automatically compose DSs and quickly generate data view to satisfy users' various data requirements (DRs).Design/methodology/approach The paper proposes an automatic DS composition and view generation approach. DSs are organized into DS dependence graph (DSDG) based on their inherent dependences, and DSs can be automatically composed using the DSDG according to user's DRs. Then, data view will be generated by interpreting the composed DS.Findings Experimental results with real cross-origination data sets show the proposed approaches have high efficiency and good quality for DS composition and view generation.Originality/value The authors propose a DS composition algorithm and a data view generation algorithm according to users' DRs.

Due to the high growth of SNS population, service scalability is one of the critical issues to be addressed. The cloud environment provides the flexible computing and storage resources for services deployment, which fits the characteristics of scalable SNS deployment. However, if the SNS related information is not properly placed, it will cause unbalance load and heavy transmission cost on the storage virtual machine (VM) and cloud data center (CDC) network. In this paper, we characterize the SNS into a graph model based on the users associations and interest correlations. The node weight represents the degree of associations, which can be indexed by the number of friends or data sources, and the link weight denotes the correlation between users/data sources. Then, based on the SNS graph, the two-step algorithm is proposed in this paper to determine the placement of SNS related data among VMs. Two k-means based clustering schemes are proposed to allocate social data in proper VM and physical servers for pre-configured VM and dynamic VM environment, respectively. The experimental example was conducted and to illustrate and compare the performance of the proposed schemes.

Scheduling workflows in cloud service environment has attracted great enthusiasm, and various approaches have been reported up to now. However, these approaches often ignored the uncertainties in the scheduling environment, such as the uncertain task start/execution/finish time, the uncertain data transfer time among tasks, the sudden arrival of new workflows. Ignoring these uncertain factors often leads to the violation of workflow deadlines and increases service renting costs of executing workflows. This study devotes to improving the performance for cloud service platforms by minimizing uncertainty propagation in scheduling workflow applications that have both uncertain task execution time and data transfer time. To be specific, a novel scheduling architecture is designed to control the count of workflow tasks directly waiting on each service instance (e.g., virtual machine and container). Once a task is completed, its start/execution/finish time are available, which means its uncertainties disappearing, and will not affect the subsequent waiting tasks on the same service instance. Thus, controlling the count of waiting tasks on service instances can prohibit the propagation of uncertainties. Based on this architecture, we develop an unceRtainty-aware Online Scheduling Algorithm (ROSA) to schedule dynamic and multiple workflows with deadlines. The proposed ROSA skillfully integrates both the proactive and reactive strategies. During the execution of the generated baseline schedules, the reactive strategy in ROSA will be dynamically called to produce new proactive baseline schedules for dealing with uncertainties. Then, on the basis of real-world workflow traces, five groups of simulation experiments are carried out to compare ROSA with five typical algorithms. The comparison results reveal that ROSA performs better than the five compared algorithms with respect to costs (up to 56 percent), deviation (up to 70 percent), resource utilization (up to 37 percent), and fairness (up to 37 percent).

With the trend of cloud computing, more and more web applications move their content to external storage services to reduce the cost of hardware and maintenance. In this paper, a novel architecture called CloudEdge for Content Delivery Network (CDN) with the storage service in cloud computing is introduced. This architecture could not only keep the loosely coupled integration between storage service and web applications but also provide better content manipulation features such as edge network content delivery, caching, secured access control, the variations of content objects generated on demand, and post-process on content objects.

With the increasing popularity of Software as a Service (SaaS) clouds, software architects face new challenges in software architecture design, e.g. efficiently integrating those chosen web services into clouds. In this respect, early quality evaluation of the designed SaaS cloud application is crucial to mitigate the risk of later architectural changes due to a violation of quality requirements (such as response time, network throughput and scalability). Architects need to navigate in a rich set of offered services, a variability of the SaaS cloud environments, which makes it difficult to predict the general approach without time-consuming implementation of application prototypes (using GraphQL, Representational State Transfer (REST), Simple Object Access Protocol (SOAP), and Application Programming Interfaces (APIs) approaches). In this paper, we present an abstract application model designed to select the details with a high level of scalability and efficient interactions between service components. This model is reusable and provides new way of thinking about cloud architecture.

In today`s world of science & technology mobile device especially Smartphones plays the role of “"Jinn" who fulfill demand of the user. Smartphone through its Internet facility allows users to access various internet based web services ubiquitously to fulfill his/her demand. Web Services can be of two types SOAP or Restful services. Many researchers have mentioned that SOAP based services are heavy to consume on mobile device despite the fact that most of the companies still uses SOAP services but answer to this challenge is Restful Web Services. Restful services also surfers from many problems like processing of work or accessing services on device due to mobile phone`s small screen size, low processing power, less memory storage capacity etc. That`s why the heavy processing work is shifted on Cloud and only client part will be on mobile device, this integration is called as Mobile Cloud Computing (MCC). In MCC important point need to consider is to improve the performance of web services in terms of speed, memory usage and reducing latency. To resolve these issues, this paper has proposed MCC architecture. We have divided this paper is into two parts, in the first part; we did comparative analysis of Web Service consumption on mobile devices using cloud and various parameters used. In second part we have proposed architecture for mobile Web Services in Cloud which compress response data using Gzip and use of web caching to reduce overhead of repeated or previously accessed Web Service.

Nowadays, digital surveillance systems are ubiquitously installed and continuously generating huge amount of video data. More often than not, it requires human to identify potential threats. To address that problem, we propose a Platform-as-a-Service (PaaS) architecture to conduct large scale video analysis in more systematic and efficient ways. In addition, the video analysis application implemented on the proposed PaaS platform provides a more user friendly interface for analyzing video footage. In this paper, we present the construction and implementation steps of a real-world video analysis service based on our previous work for a citywide security service. We successfully integrated over 25 thousands of surveillance devices throughout the city. The integrations of the NVR/DVRs, license plate number recognition engine, and the camera health monitoring engine will be demonstrated. Finally, we conclude with discussions on why we believe the proposed solution has the potential to become the standard for deploying intelligent video surveillance network.