Deep Reinforcement Learning for Cooperative Edge Caching in Future Mobile Networks

360º video is an essential component of VR/AR/MR systems that provides immersive experience to the users. However, 360º video is associated with high bandwidth requirements. The required bandwidth can be reduced by exploiting the fact that users are interested in viewing only a part of the video scene and that users request viewports that overlap with each other. Motivated by the findings of our recent works where the benefits of caching video tiles at edge servers instead of caching entire 360º videos were shown, in this paper, we introduce the concept of virtual viewports that have the same number of tiles with the original viewports. The tiles forming these viewports are the most popular ones for each video and are determined by the users' requests. Then, we propose a proactive caching scheme that assumes unknown videos' and viewports' popularity. Our scheme determines which videos to cache as well as which is the optimal virtual viewport per video. Virtual viewports permit to lower the dimensionality of the cache optimization problem. To solve the problem, we first formulate the content placement of 360º videos in edge cache networks as a Markov Decision Process (MDP), and then we determine the optimal caching placement using the Deep Q-Network (DQN) algorithm. The proposed solution aims at maximizing the overall quality of the 360º videos delivered to the end-users by caching the most popular 360º videos at base quality along with a virtual viewport in high quality. We extensively evaluate the performance of the proposed system and compare it with that of known systems such as Least Frequently Used (LFU), Least Recently Used (LRU), First-In-First-Out (FIFO), over both synthetic and real 360º video traces. The results reveal the large benefits coming from proactive caching of virtual viewports instead of the original ones in terms of the overall quality of the rendered viewports, the cache hit ratio, and the servicing cost.

Support vector machines (SVMs) can solve structured multioutput learning problems such as multilabel classification, multiclass classification, and vector regression. SVM training is expensive, especially for large and high-dimensional data sets. The bottleneck of the SVM training often lies in the kernel value computation. In many real-world problems, the same kernel values are used in many iterations during the training, which makes the caching of kernel values potentially useful. The majority of the existing studies simply adopt the least recently used (LRU) replacement strategy for caching kernel values. However, as we analyze in this article, the LRU strategy generally achieves high hit ratio near the final stage of the training but does not work well in the whole training process. Therefore, we propose a new caching strategy called EFU (less frequently used), which replaces the EFU kernel values that enhance least frequently used (LFU). Our experimental results show that EFU often has 20% higher hit ratio than LRU in the training with the Gaussian kernel. To further optimize the strategy, we propose a caching strategy called hybrid caching for the SVM training (HCST), which has a novel mechanism to automatically adapt the better caching strategy in different stages of the training. We have integrated the caching strategy into ThunderSVM, a recent SVM library on many-core processors. Our experiments show that HCST adaptively achieves high hit ratios with little runtime overhead among different problems including multilabel classification, multiclass classification, and regression problems. Compared with other existing caching strategies, HCST achieves 20% more reduction in training time on average.

Access to the internet and WWW is growing extensively, which results in heavy network traffic. To reduce the network traffic proxy server is used. Proxy server reduces the load of server. If the cache replacement algorithm of proxy server's cache is efficient then proxy server will be helpful to reduce the network traffic in more efficient manner. In this paper we are considering proxy server cache to be Level 1 (L1) cache and storage cache of proxy server to be Level 2 (L2) cache. For collecting the real trace CC proxy server is used in an organization. Log of proxy server gives the information of various URLs accessed by various clients with time. For performing experiments various URLs were given a numeric identity. This paper proposes an efficient replacement algorithm on L1 for proxy server. The replacement algorithms taken into consideration are Least Recently Used (LRU), Least Frequently Used (LFU), First In First Out (FIFO). Access Pattern of L1 and L2 are different as if the desired page is not on L1 then it accesses to L2. Thus L1 is having better temporal locality than L2. Thus the replacement algorithm which is giving efficient results for L1 may not be suitable for L2. This paper also proposes a preeminent pair of replacement algorithms for L1 and L2 cache for proxy server.

Caching on the edge has been recognized as an effective solution to tackle the backhaul constraint of network densification. However, most related works ignored user mobility in wireless networks, which is unreasonable under the background of network densification. For a more flexible and context-aware caching decision, the concept of caching on the edge can be extended to mobile edge computing (MEC) that enables computation and storage resources at mobile edge networks. With MEC servers deployed on base stations, a huge amount of collected radio access network context data can be analyzed and utilized to render a caching scheme adaptive to user’s context-aware information. In this regard, a novel mobility-aware coded probabilistic caching scheme is proposed for MEC-enabled small cell networks (SCNs). Different from previous mobility-aware caching schemes, user mobility and distributed storage are incorporated into a conventional probabilistic caching scheme, with the aim of throughput maximization. Based on stochastic geometry theory and a modified mobility model of discrete random jumps, the explicit expression of throughput is derived. Due to the complexity of the expression, two light-weight heuristic algorithms are provided to numerically obtain the optimal solutions. Moreover, a significant trade-off among the gains of mobility diversity, content diversity, and channel selection diversity is discussed, and we further numerically analyze how such a trade-off is influenced by user mobility, content popularity, and backhaul capacity, with some fundamental insights into the application of the proposed scheme in MEC-enabled SCNs. The superiority of our proposed scheme is demonstrated by the comparisons with the classical M most popular caching scheme and the conventional probabilistic caching scheme. Numerical results show that our proposed caching scheme achieves higher throughput than those of the other two, especially when users of intense mobility request contents, of which the popularity profile is not skewed, in MEC-enabled SCNs with poor backhaul capacity, indicating that the proposed caching scheme is a promising solution for network densification.

As a main driver of 5G, the future mobile network is going to create a hyper‐connected Internet of Everything (IoE) world which requires the network to be scalable, versatile, and energy smart. The convergence of communication and computing is a promising solution of IoE while the introduction of fog/edge computing makes this possible in the future mobile networks. This article provides a comprehensive survey of the fog/edge‐computing‐enabled mobile network. Specifically, the motivation of introducing fog/edge computing, the standardization of fog/edge computing in 5G, and the mobile communication network architecture with fog/edge computing are reviewed. The challenges such as the scalability and extreme heterogeneity of resource management in IoE, and the advantages of using fog/edge‐based architecture, are also discussed. Under the fog/edge‐computing‐driven mobile communication network architecture, a distributed resource allocation mechanism based on economics is presented. The economics models and analysis study the underlying, and possibly conflicting objectives and incentives of various stakeholders, with pricing solutions providing a way for the optimization of objectives like social welfare, profit, or fairness. As such, it can be a potential powerful tool in solving the aforementioned challenges. While pricing for resource allocation has a vast existing literature, the analysis and schemes cannot be implemented directly due to the unique characteristics of 5G cloud‐fog/edge networks, hence providing a vast scope for potential research. In light of this, we finally outline some unique characteristics to consider when applying economics to aid resource allocation in 5G cloud‐fog/edge networks.

The explosion of traffic caused by the rapid growth of multimedia services of Internet of Vehicles (IoV) has brought heavy load to mobile networks. The edge caching of the Internet of vehicles is considered as a promising technology. When the existing content caching strategy is used in the vehicle network, it faces the challenge of high content caching delay caused by the high-speed mobility of vehicle users and insufficient social relations. To address these challenges, this paper proposes a Cooperative Edge Caching Scheme based on Mobility Prediction and Society Aware (CCMPSA). In this strategy, the Long Short-Term Memory (LSTM) network is used to predict the location of the vehicle at the next moment, the vehicle cache nodes are selected according to the social relationship reflected by the similarity of interest and communication probability between the vehicles, and the dynamic decision of the content cache problem is realized by deep reinforcement learning. The simulation results show that the performance of the proposed strategy is better than random caching and non-cooperative caching algorithms, and it not only reduces the content transmission delay and improves the cache hit ratio, but also improves the experience quality of the whole system.KeywordsCaching decisionsDeep reinforcement learningThe edge cachingThe long short-term memory networkInternet of Vehicles

Vehicular Edge Computing (VEC) is a promising paradigm to enable huge amount of multimedia content to be cached in proximity to vehicles. Since vehicles are equipped with a certain amount of caching resource, they can be regarded as edge nodes to expand the caching capacity of the network edge. However, with much sensitive personal information, vehicles may be not willing to cache their content to an untrusted vehicle. Permission blockchain has the potential to address such an issue. In this paper, we utilize permissioned blockchain to design a secure content caching scheme between vehicles. Since high mobility of vehicles makes a dynamic caching environment, we exploit deep reinforcement learning approach to design the content caching scheme. Moreover, we propose a new block verifier selection metric, Proof-of-Utility (PoU), to enable a lightweight permissioned blockchain. Security analysis shows that our proposed blockchain empowered content caching can achieve security and privacy protection. Numerical results based on the Uber dataset indicate the DRL-inspired content caching scheme significantly outperforms two benchmark policies.

We consider a mobile network consisting of both the wireless access network and the backhaul network. All base stations in the access network and gateways in the backhaul network are equipped with caches, so that routing costs for serving content requests can be reduced by caching the requested content items closer to the users. In this case, user association in the wireless access network must be aware of both the quality of wireless channels and the content caching strategy. In this paper, we propose a framework that jointly optimizes wireless user association and content caching in both access and backhaul networks. The resulting problem is NP-hard. We propose a polynomial-time algorithm based on convex approximation and pipage rounding that produces a solution within a constant factor of 1 − 1/e from the optimal. Simulation results show that the proposed joint algorithm outperforms schemes that combine cache-independent user association methods with traditional caching strategies (e.g. LRU) in terms of minimizing the aggregate routing cost and backhaul traffic while achieving a high data sum rate in the access network.

Content caching in mobile edge networks has stirred up tremendous research attention. However, most existing studies focus on predicting content popularity in mobile edge servers (MESs). In addition, they overlook how the content is cached, especially how to cache the content with user devices. In this paper, we propose CMU, a three-layer (Cloud-MES-Users) content caching framework and investigate the performance of different caching strategies under this framework. A user device who has cached the content can offer the content sharing service to other user devices through device-to-device communication. In addition, we prove that optimizing the transmission performance of CMU is an NP-hard problem. We provide a solution to solve this problem and describe how to calculate the number of distributed caching nodes under different parameters, including time, energy and storage. Finally, we evaluate CMU through a numerical analysis. Experiment results show that content caching with user devices could reduce the requests to Cloud and MESs, and decrease the content delivery time as well.

Content caching at base stations (BSs) is a promising technique for future wireless networks by reducing network traffic and alleviating server bottleneck. However, in practice, the content popularity distribution may change with spatio-temporal variation but be unknown for BSs, which is an intractable obstacle for efficient caching strategy design. In this paper, considering unknown popularity distribution, we explore the content caching problem by jointly optimizing the content caching in cooperative BSs, content sharing among BSs, and cost of content retrieving. We tackle the problem from a multi-armed bandit learning perspective, where the learning of the popularity distribution is incorporated with the content caching and sharing process. Specifically, we first propose a centralized algorithm by employing a semidefinite relaxation approach, and we prove that this centralized algorithm learns efficient caching by deriving a sub-linear learning regret bound. To further reduce computational complexity, we propose a distributed algorithm based on alternating direction method of multipliers, where each BS only solves their own problems by exchanging local information with neighbor BSs. Extensive simulation results show the effectiveness of the proposed algorithms in terms of learning content popularity distributions of individual BSs, offloading traffic from the content server, and reducing cost of content retrieving.

In this paper, content caching is considered in a software-defined hyper-cellular network (SD-HCN) with capacity-limited backhaul connections. To achieve efficient content caching and delivery at the network edge, an analytical framework of minimizing the average content provisioning cost of SD-HCN, e.g., latency, bandwidth, and so on, is first formulated subjected to a sum storage capacity constraint. An optimal solution to this problem requires a joint design of storage allocation and content placement at the centralized control base station (CBS) and distributed traffic base stations (TBSs), which is NP-hard in general. To provide insights, a baseline non-cooperative caching strategy is first introduced between the CBS and TBSs. Then, an efficient cooperative edge caching strategy is proposed by leveraging the vertical cooperation between the CBS and TBSs, and horizontal cooperation between the TBSs. Analytical results demonstrate that the content provisioning cost of SD-HCN is significantly reduced by using the analytically obtained optimal storage allocation between the CBS and TBSs, and the proposed cooperative edge caching strategy always outperforms the non-cooperative caching strategy. Furthermore, by switching between the vertical and horizontal cooperative caching modes, extra performance gains can be achieved by the proposed cooperative edge caching strategy.

The development of Network Function Virtualization (NFV) and Software Defined Networks (SDN) standards is an opportunity for Mobile Network Operators (MNOs) to deploy Content Delivery Network (CDN) functionalities into the mobile network edge, such as Base Stations (BSs). In this paper, we investigated the content placement problem for the BS-based wireless CDN system. We call the storage resources implemented on BSs as storage helpers. Due to the limited helper storage capacity and the limited user population served per BS, helpers exhibit low hit ratio comparing to traditional CDN edge servers serving a wide area. Then, cooperation is a suitable means to enhance the performance of the wireless CDN system. We propose that BSs close to each other cooperate in replicating content and replying user requests. We formulate the optimum content placement problem to minimize the traffic pressure on mobile network gateways, and show the problem complexity is NP-Hard. We then transform the problem into a multiple-Maximum Weighted Independent Set problem, and propose a heuristic algorithm. The evaluation shows that the hit ratio is improved by our algorithm comparing to the traditional Least Frequently Used (LFU) policy without cooperation.

Increasing the speed of cache simulation to obtain hit/miss rates enables performance estimation, cache exploration for embedded systems and energy estimation. Previously, such simulations, particularly exact approaches, have been exclusively for caches which utilize the least recently used (LRU) replacement policy. In this paper, we propose a new, fast and exact cache simulation method for the First In First Out(FIFO) replacement policy. This method, called DEW, is able to simulate multiple level 1 cache configurations (different set sizes, associativities, and block sizes) with FIFO replacement policy. DEW utilizes a binomial tree based representation of cache configurations and a novel searching method to speed up simulation over single cache simulators like Dinero IV. Depending on different cache block sizes and benchmark applications, DEW operates around 8 to 40 times faster than Dinero IV. Dinero IV compares 2.17 to 19.42 times more cache ways than DEW to determine accurate miss rates.

The increasing popularity of photo sharing in social networking service (SNS) complicates the challenge of storing and transmitting large photo data for SNS providers. Distributed web caches are generally used by SNS providers to improve data transmission performance effectively. Two critical factors affect the efficiency of web caches: storage media and replacement algorithms. Solid-state drive (SSD) is often used in web caches due to its good performance. Random write performance and SSD endurance are directly affected by the characteristic of write amplification, this characteristic is also related to replacement algorithms. The least recently used (LRU) algorithm effectively computes cache hit ratio but induces severe write amplification. First in first out (FIFO) method minimizes write amplification, whereas a weak hit ratio increases bandwidth consumption. In this paper, we present a hybrid replacement strategy, namely, F-SkLRU. This strategy facilitates a good trade-off between performance and the cost of web cache. F-SkLRU takes advantage of sequential writing to reduce SSD wear. This strategy also consolidates multiple factors to improve hit ratio. Simulation results show that F-SkLRU can reduce bandwidth consumption by 14.11% to 23.4% in comparison with that of FIFO and LRU. The proposed strategy can also reduce the cost of SSD by up to 300% more than LRU can in extreme situations.

Cooperative caching in the device-to-device (D2D) networks is an effective solution to the problem of offloading data traffic at peak times. However, one of the main challenges of cooperative caching is the selfish nature of the user, and some users only cache what they are interested in and don't care about other people's needs. In this paper, we investigate a physical-social-based cooperative caching framework, considering the physical-social tie as a non-monetary incentive to motivate the data cache and share between users by using D2D communication. Specifically, we first define a novel physical-social tie, combining the physical factors and social relationship. After that, we formulate two mathematical programming models with or without the consideration of indirect physical-social tie, to find an optimal caching strategy with the target of maximizing the social group utility. Moreover, we adopt Six Degrees of Separation to analyze the indirect physical-social tie between users when these social users with direct physical-social tie cannot meet the date request of the user, and then devise an indirect social group formation algorithm (ISGFA) to obtain the indirect social group of the user. Finally, simulation results shows that the caching scheme obtained from our model is efficient and outperforms other caching schemes that without considering the physical-social tie.