Blockchain-Enabled Storage Resource Trading for Collaborative Edges

Network slicing of multi-access edge computing (MEC) resources is expected to be a pivotal technology to the success of 5G networks and beyond. The key challenge that sets MEC slicing apart from traditional resource allocation problems is that edge nodes depend on tightly-intertwined and strictly-constrained networking, computation and storage resources. Therefore, instantiating MEC slices without incurring in resource over-provisioning is hardly addressable with existing slicing algorithms. The main innovation of this paper is Sl-EDGE, a unified MEC slicing framework that allows network operators to instantiate heterogeneous slice services (e.g., video streaming, caching, 5G network access) on edge devices. We first describe the architecture and operations of Sl-EDGE, and then show that the problem of optimally instantiating joint network-MEC slices is NP-hard. Thus, we propose near-optimal algorithms that leverage key similarities among edge nodes and resource virtualization to instantiate heterogeneous slices 7.5x faster and within 25% of the optimum. We first assess the performance of our algorithms through extensive numerical analysis, and show that Sl-EDGE instantiates slices 6x more efficiently then state-of-the-art MEC slicing algorithms. Furthermore, experimental results on a 24-radio testbed with 9 smartphones demonstrate that Sl-EDGE provides simultaneously highly-efficient slicing of joint LTE connectivity, video streaming over WiFi, and ffmpeg video transcoding.

Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning

Electricity trading pricing among prosumers with game theory-based model in energy blockchain environment

Convolutional neural networks (CNNs) have been applied in numerous Internet of Things (IoT) devices for multifarious downstream tasks. However, with the increasing amount of data on edge devices, CNNs can hardly complete some tasks in time with limited computing and storage resources. Recently, filter pruning has been regarded as an effective technique to compress and accelerate CNNs, but existing methods rarely prune CNNs from the perspective of compressing high-dimensional tensors. In this article, we propose a novel theory to find redundant information in 3-D tensors, namely, quantified similarity between feature maps (QSFM), and utilize this theory to guide the filter pruning procedure. We perform QSFM on data sets (CIFAR-10, CIFAR-100, and ILSVRC-12) and edge devices and demonstrate that the proposed method can find the redundant information in the neural networks effectively with comparable compression and tolerable drop of accuracy. Without any fine-tuning operation, QSFM can compress ResNet-56 on CIFAR-10 significantly (48.7% FLOPs and 57.9% parameters are reduced) with only a loss of 0.54% in the top-1 accuracy. For the practical application of edge devices, QSFM can accelerate MobileNet-V2 inference speed by 1.53 times with only a loss of 1.23% in the ILSVRC-12 top-1 accuracy.

Serverless edge computing environments use lightweight containers to run IoT services on a need basis i.e only when a service is requested. These containers experience a cold start up latency when starting up. One probable solution to reduce the startup delay is container caching on the edge nodes. Edge nodes are nodes that are closer in proximity to the IoT devices. Efficient container caching strategies are required since the resource availability on these edge devices is limited. Because of this constraint on resources, the container caching strategies should also take proper resource utilization into account. This project tries to further improve the startup latency and provides explanations of a resulting caching strategy by using Explainable Reinforcement Learning (XRL). This project proposes a method that uses Deep Reinforcement Learning (DRL) to learn a policy to efficiently perform container caching by maximizing the expected cumulative discounted reward. The proposed method uses a probability distribution function to distribute IoT service requests to the other edge devices by taking advantage of the heterogeneous and distributed character of edge computing environments. An XRL method called the action-influence model will be integrated into the DRL framework so that a service provider of the edge computing service can understand the logic behind the caching and further improve the performance. A comprehensive analysis of the impact of caching decisions on overall performance of the DRL algorithm is provided.

Federated learning (FL) enables privacy-preserving distributed machine learning by training models on edge client devices using their local data without revealing their raw data. In edge environments, various applications require different neural network models, making it crucial to perform joint training of multiple models on edge devices, known as multi-task FL. While existing multi-task FL approaches enhance resource utilization on edge devices through adaptive resource configuration or client selection, optimizing either of these aspects alone may lead to suboptimality. Therefore, in this paper, we explore a joint client selection and resource configuration method called JCSRC for multi-task FL, aiming to maximize energy efficiency in environments with limited computation and communication resources and heterogeneous client devices. Firstly, we formalize this problem as a mixed-integer nonlinear programming problem considering all these characteristics and prove its NP-hardness. To address this problem, we first design a multi-agent reinforcement learning (MARL)-based client selection method that selects appropriate clients for each task to train their models. The MARL method makes client selection decisions based on the clients’ data quality, energy efficiency, communication, and computation capacity to ensure fast convergence and energy efficiency. Then, we design a particle swarm optimization (PSO)-based resource configuration scheme that configures appropriate computation and bandwidth resources for each task on each client. The PSO scheme makes resource configuration decisions based on theoretically derived optimal CPU frequency and bandwidth to achieve high energy efficiency. Finally, we carry out extensive simulations and testbed-based experiments to validate our proposed JCSRC. The results demonstrate that, in comparison to state-of-the-art solutions, JCSRC can save energy consumption by up to 59% to achieve the target accuracy.

With the commercialization of 5G, the era of hyper-connectivity is accelerating its evolution. The cloud-edge collaborative networks (CECN) has become a key infrastructure supporting the intelligent society. Its open and dynamic nature has also introduced continuously evolving security threats. Traditional static defense has limitations in responding to dynamic attacks and achieving cross-domain resource collaboration. We propose a deception defense method based on two-layer game and multi-agent reinforcement learning. We analyze the network state evolution process. We construct a two-layer architecture that integrates cooperative game and Stackelberg game, and we design a strategy solving method based on multi-agent reinforcement learning (MARL) to achieve adaptive optimization of defense strategy. The simulation results show that the proposed method can effectively improve the overall defense effectiveness and maintain fairness within the defense alliance.

Imposter devices pose serious threats. The majority of low-cost edge devices can easily be counterfeited or cloned; the supply chain is insufficiently secure. Reliability of deployed devices can be called into question simply because they might be counterfeit or cloned. It is a must to identify edge devices' sourcing uniquely and verify their validity periodically at runtime. We integrate blockchain technology to authenticate resource-constrained, low-cost edge devices. We use SRAM-based physically unclonable functions (PUFs)to generate unique “digital fingerprints” (device IDs). Registered manufacturers upload a cryptographic hash of each device ID to a “globally accessible” blockchain instance (key-value store or smart contract). While registering/designating a device locally, the end-user verifies whether the hash is present in that blockchain. We utilize a “locally permissioned” blockchain infrastructure (which is still a globally managed blockchain or, in future, a sidechain)to authenticate edge devices for a defense-in-depth approach. Devices can authenticated periodically to prevent device cloning. Target environments can be large and have varied trust among users and lack a specific perimeter; this “local” blockchain methodology is thus pertinent, especially since blockchains gain security over time. Our approach reduces the potential for classes of information leakage and types of sabotage in a critical infrastructure or large-scale deployment (such as a smart city)arising from imposter devices. This methodology protects against such imposters in mobile settings within an IoT infrastructure too.

Edge caching is a key technology to accelerate the content delivery and to mitigate the network congestion. As a real-world example, Microsoft has been providing the edge caching service with a usage-based pricing mechanism since 2009. However, the challenge of designing a better pricing mechanism lies in balancing the finer grain size of the mechanism and the complexity of business operations it required. Our work proposes a novel location dependent pricing mechanism for the edge caching market which both improves the Internet service provider's (ISP) revenue and the social welfare. First, we introduce the location dependent pricing mechanism (LDP) that offers location-specific price, considering the popularity differences between the edge nodes(e.g. base stations, WiFi access points) at different locations. We capture the interactions between the content providers (CPs) and the ISP as a two-stage Stackelberg game and derive the CP's best responses and the ISP's optimal pricing strategy. Then we provide a partial location dependent pricing scheme (PLDP) which limits the number of price tiers to provide a balance between the optimality and economic viability. Finally, through extensive numerical simulation, we find that the LDP (or PLDP) improves the revenue of the ISP by 31%↑50% (23%↑%35) and also improves the social welfare by 46% (%32) on average, compared to the uniform pricing scheme.

For the task under Markov Decision Process, this paper presents a novel multiagent Reinforcement Learning (RL) with perception and conversion action mechanism that learning agents observe adversary agent and convert adversarial action to learning agents' corresponding action as observing state variation incurred by the adversary agent in the task environment during learning processes. Meanwhile, this paper surveys inexpensive communication ways among learning agents utilizing both the direct communication and the indirect media communication to realize agents' cooperation. The direct communication is realized by sharing sensation; the indirect media communication is realized by updating reinforcement values on the common environment observation. Then, a multiagent RL algorithm, Q-ac multiagent RL method, is proposed. By perception and conversion action, the learning agents extend learning episodes and derive more observation by less action. The direct communication enhances agents' observation ability to the environment, and the indirect media communication improves agents' ability deriving the optimal action policy. The simulation results on hunter game demonstrate the efficiency of the proposed method.

Interactive segmentation has recently been explored to effectively and efficiently harvest high-quality segmentation masks by iteratively incorporating user hints. While iterative in nature, most existing interactive segmentation methods tend to ignore the dynamics of successive interactions and take each interaction independently. We here propose to model iterative interactive image segmentation with a Markov decision process (MDP) and solve it with reinforcement learning (RL) where each voxel is treated as an agent. Considering the large exploration space for voxel-wise prediction and the dependence among neighboring voxels for the segmentation tasks, multi-agent reinforcement learning is adopted, where the voxel-level policy is shared among agents. Considering that boundary voxels are more important for segmentation, we further introduce a boundary-aware reward, which consists of a global reward in the form of relative cross-entropy gain, to update the policy in a constrained direction, and a boundary reward in the form of relative weight, to emphasize the correctness of boundary predictions. To combine the advantages of different types of interactions, i. e., simple and efficient for point-clicking, and stable and robust for scribbles, we propose a supervoxel-clicking based interaction design. Experimental results on four benchmark datasets have shown that the proposed method significantly outperforms the state-of-the-arts, with the advantage of fewer interactions, higher accuracy, and enhanced robustness.

In a two-stage Stackelberg game, depending on the leader’s information about the choice of the follower among his optimal responses, one can associate different types of mathematical problems. We present formulations and solution concepts for such problems, together with their possible roles in bilevel optimization, and we illustrate the crucial issues concerning these solution concepts. Then, we discuss which of these issues can be positively or negatively answered and how managing the latter ones by means of two widely used approaches: regularizing the set of optimal responses of the follower, via different types of approximate solutions, or regularizing the follower’s payoff function, via the Tikhonov or the proximal regularizations. The first approach allows to define different kinds of regularized problems whose solutions exist and are stable under perturbations assuming sufficiently general conditions. Moreover, when the original problem has no solutions, we consider suitable regularizations of the second-stage problem, called inner regularizations, which enable to construct a surrogate solution, called viscosity solution, to the original problem. The second approach permits to overcome the non-uniqueness of the follower’s optimal response, by constructing sequences of Stackelberg games with a unique second-stage solution which approximate in some sense the original game, and to select among the solutions by using appropriate constructive methods.

The introduction of micro grids (MGs) into the utility grid poses new challenges in the energy management design due to the intermittent characteristics of renewable energy sources and limited storage capacity. In this paper, we proposed a distributed energy management algorithm by taking into consideration the interactions and interconnections among utility companies, MGs, and customers. We model the energy management problem as a two-stage Stackel berg game, in which utility companies and MGs are game leaders, and customers are game followers. Utility companies and MGs make decisions about what price to offer their electricity to customers. Customers adjust their electricity procurement amounts based on the prices offered by utility companies and MGs. We prove that a Nash equilibrium exists in the proposed two-stage Stackel berg game, and the optimum solutions obtained by the distributed energy management algorithm is exactly the Nash equilibrium. We have analyzed and verified the relationships among utility functions, electricity prices, electricity demands, electricity procurement amounts, and pollutant parameters through computer simulations. We have also compared the performance of the proposed distributed algorithm with the centralized algorithm under different simulation conditions.

This article investigates the multirobot efficient search (MuRES) for a nonadversarial moving target problem from the multiagent reinforcement learning (MARL) perspective. MARL is deemed as a promising research field for cooperative multiagent applications. However, one of the main bottlenecks of applying MARL to the MuRES problem is the nonstationarity introduced by multiple learning agents. With learning agents simultaneously updating their policies, the environment cannot be modeled as a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stationary</i> Markov decision process, which results in the inapplicability of fundamental reinforcement learning techniques such as deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -network and policy gradient (PG). In view of that, we adopt the centralized training and decentralized execution scheme and thereby propose a cross-entropy regularized policy gradient (CE-PG) method to train the learning agents/robots. We let the robots <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">commit</i> to a predetermined policy during execution, collect the trajectories, and then perform centralized training for the corresponding policy improvement. In this way, the nonstationarity problem is overcome, in that the robots do not update their policies during execution. During the centralized training stage, we improve the canonical PG method to consider the interactions among robots by adding a cross-entropy regularization term, which essentially functions to “disperse” the robots in the environment. Extensive simulation results and comparisons with state of the art show CE-PG's superior performance, and we also validate the algorithm with a real multirobot system in an indoor moving target search scenario.

The evolution toward 6G Computing Power Networks (CPN) aims to deeply integrate multi-tier computing resources across Cloud, Edge, and end devices. However, the significant heterogeneity of computing resources, characterized by varying hardware architectures such as CPUs, GPUs, and NPUs, coupled with the time-varying network topology caused by terminal mobility, poses severe challenges to realizing efficient integrated scheduling that satisfies Quality of Service (QoS). To address spatiotemporal mismatches between task requirements and hardware architectures, this paper proposes an integrated scheduling method combining Discrete Time-Varying Graph (DTVG) construction with Multi-Agent Reinforcement Learning (MARL). Specifically, we model the dynamic interaction between mobile tasks and heterogeneous nodes as a DTVG to capture spatiotemporal evolution and employ a QMIX-based algorithm to enable collaborative decision-making among distributed agents. Simulation results demonstrate that the proposed approach effectively solves the joint optimization problem of heterogeneous resource matching and dynamic path planning, significantly outperforming traditional baselines in terms of resource utilization and average latency. This study confirms that incorporating graph-theoretic modeling with reinforcement learning offers a robust solution for the complex coupling of communication and computation in dynamic 6G networks.