An iterative stochastic algorithm based on distributed learning automata for finding the stochastic shortest path in stochastic graphs

A sampling method based on distributed learning automata for solving stochastic shortest path problem

The shortest path between the start node and end node plays an important role in city’s road traffic network analysis system. The basic ant colony system algorithm which is a novel simulated evolutionary algorithm is studied to solve the shortest path problem. But the basic ant colony system algorithm is easy to run into the local optimum solution for shortest path. In order to solve the problem, the improved ant colony system algorithm is proposed. The improvement methods for selection strategy, local search, and information quantity modification of basic ant colony system are discussed in detail. The experiments are done in Beijing road network in China. The results of experiments show that comparing with the basic ant colony algorithm, the improved algorithm can easily converge at the global optimum for the shortest path.

This paper presents the first Learning Automaton-based solution to the dynamic single source shortest path problem. It involves finding the shortest path in a single-source stochastic graph topology where there are continuous probabilistic updates in the edge-weights. The algorithm is significantly more efficient than the existing solutions, and can be used to find the "statistical" shortest path tree in the "average" graph topology. It converges to this solution irrespective of whether there are new changes in edge-weights taking place or not. In such random settings, the proposed learning automata solution converges to the set of shortest paths. On the other hand, the existing algorithms will fail to exhibit such a behavior, and would recalculate the affected shortest paths after each weight-change. The important contribution of the proposed algorithm is that all the edges in a stochastic graph are not probed, and even if they are, they are not all probed equally often. Indeed, the algorithm attempts to almost always probe only those edges that will be included in the shortest path graph, while probing the other edges minimally. This increases the performance of the proposed algorithm. All the algorithms were tested in environments where edge-weights change stochastically, and where the graph topologies undergo multiple simultaneous edge-weight updates. Its superiority in terms of the average number of processed nodes, scanned edges and the time per update operation, when compared with the existing algorithms, was experimentally established. The algorithm can be applicable in domains ranging from ground transportation to aerospace, from civilian applications to military, from spatial database applications to telecommunications networking.

The integration of the social network concepts and IoT has led to the creation of a new concept called Social IoT, wherein objects or things can interact and provide services to one another. Efficient and distributed service navigation in such systems is essential due to the potentially huge number of services that are provided. This paper proposes a Distributed Learning Automata-based algorithm called DLA_N, which aims at finding service-providers (i.e., objects) with high popularity or influence. The main idea is that an object can use its friends or friends of its friends to search for the desired service provider. Taking into account the SIoT’s graph properties (i.e., the topology of the network), we define a new centrality metric that indicates the importance degree of a node in SIoT. Embedding a Learning Automata in each object, a distributed LA approach is proposed for the selection of the most influential nodes in the network. Starting from a requesting object, the proposed DLA_N algorithm learns to select a path containing objects with a high centrality metric. The distributed nature of our navigation results in high scalability and low navigation time. The results of performance evaluation indicate that the proposed method outperforms existing methods in the literature.

Many factors influence the connection states between nodes of wireless sensor networks, such as physical distance, and the network load, making the network’s edge length dynamic in abundant scenarios. This dynamic property makes the network essentially form a graph with stochastic edge lengths. In this paper, we study the stochastic shortest path problem on a directional graph with stochastic edge lengths, using reinforcement learning algorithms. we regard each edge length as a random variable following unknown probability distribution and aim to find the stochastic shortest path on this stochastic graph. We evaluate the performance of path-finding algorithms using regret, which represents the cumulative reward difference between the practical path-finding algorithm and the optimal strategy that chooses the global stochastic shortest path every time. We model the path-finding procedure as a Markov decision process and propose two online path-finding algorithms: Q SSP algorithm and SARSA SSP algorithm, both combined with specifically-devised average reward mechanism. We justify the convergence property and correctness of the proposed algorithms theoretically. Experiments conducted on two benchmark graphs illustrate the superior performance of the proposed Q SSP algorithm which outperforms the SARSA SSP algorithm and other competitive algorithms about the regret metric.

In view of the disadvantages of the basic ant colony algorithm in solving non-deterministic polynomial problems, such as slow convergence rate and local optimal solution. In this paper, the updating rules of pheromone and the updating strategy of target city in ant colony algorithm are improved, and an improved ant colony algorithm (IACA) is proposed. In addition, it is applied to the shortest traversal path search of travel salesman problem. Taking 33 provincial capitals and berlin52 problems as test objects, this paper compares the search efficiency of the proposed algorithm with that of the basic ant colony algorithm. It can be seen from the simulation results that IACA has a faster convergence rate than the basic ant colony algorithm. IACA is obviously superior to the basic ant colony algorithm in terms of convergence and the quality of the optimal solution. It provides a new choice for solving similar optimization problems.

Reconfiguring two shortest paths in a graph means modifying one shortest path to the other by changing one vertex at a time, so that all the intermediate paths are also shortest paths. This problem has several natural applications, namely: (a) revamping road networks, (b) rerouting data packets in a synchronous multiprocessing setting, (c) the shipping container stowage problem, and (d) the train marshalling problem. When modelled as graph problems, (a) is the most general case while (b), (c) and (d) are restrictions to different graph classes. We show that (a) is intractable, even for relaxed variants of the problem. For (b), (c) and (d), we present efficient algorithms to solve the respective problems. We also generalise the problem to when at most k (for some k >= 2) contiguous vertices on a shortest path can be changed at a time.

Finding shortest distances and paths have always been a crucial area of study. In this article, we have proposed a linear time pre-processing method for optimization of shortest path and distance algorithms, for an undirected weighted graph with non-negative edges. In this approach, certain parts of a graph are grouped together as subgraphs termed as optimized deterministic routing areas (ODRAs). Each ODRA has a unique representative element called optimized proxy. The input graph is reduced by replacing ODRAs with optimized proxies and the new reduced graph is used for the shortest path and distance queries. This is a linear time pre-processing stage which helps in significant decrease in computation time for finding shortest distances and paths in graphs. The lower bound and upper bound of time complexity of the proposed algorithm is O(n/2) and O(n) respectively.

So many algorithms have been proposed to solve the shortest path in road networks, in this paper, an algorithm is developed to solve shortest route problems. The algorithm is being demonstrated through solving of various network problems. The principle of the algorithm consist in transforming the graph into a tree by means of arc and node replication, thereby expanding outwards from the source node considering all possible paths up to the destination node. The objective is to develop a method that can be applied in directed and non-directed graphs.

In this paper, a new structure for cooperative learning automata called extended learning automata (eDLA) is introduced. Based on the new structure, an iterative randomized heuristic algorithm using sampling is proposed for finding an optimal subgraph in a stochastic edge-weighted graph. Stochastic graphs are graphs in which the weights of edges have an unknown probability distribution. The proposed algorithm uses an eDLA to find a policy that leads to a subgraph that satisfy some restrictions such as minimum or maximum weight (length). At each stage of the proposed algorithm, the eDLA determines which edges should be sampled. The proposed eDLA-based sampling method may reduce unnecessary samples and hence decrease the time required for finding an optimal subgraph. It is shown that the proposed method converges to an optimal solution, the probability of which can be made arbitrarily close to 1 by using a sufficiently small learning parameter. A new variance-aware threshold value is also proposed that can significantly improve the convergence rate of the proposed eDLA-based algorithm. It is further shown that our algorithm is competitive in terms of the quality of the solution.

In various applications where the problem domain can be modeled into graphs, the shortest path computation in the graph is an indispensable challenge. In applications like online social networks and shortest route computation problems, the size of the graph is so large; the number of nodes have become close to hundreds of billions. Shortest path graph algorithms like SSSP (Single Source Shortest Path) and APSP (All Pairs Shortest Path) have low arithmetic intensity and irregular memory access patterns. The GPU implementation of many arithmetic and logical problems exceed the performance of the CPU system implementations. There is an increasing need for faster computation of shortest path in graphs for various applications. The objective of the work is to demonstrate that GPUs can efficiently perform shortest path computations on undirected weighted graphs considering space efficiency. For the implementation, GPUs supporting CUDA (Compute Unified Device Architecture) programming have been used. Additionally a space efficient approach called Triangular Matrix Method has been used.

Consider a complete graph Kn with edge weights drawn independently from a uniform distribution U(0, 1). The weight of the shortest (minimum‐weight) path P1 between two given vertices is known to be , asymptotically. Define a second‐shortest path P2 to be the shortest path edge‐disjoint from P1, and consider more generally the shortest path Pk edge‐disjoint from all earlier paths. We show that the cost Xk of Pk converges in probability to uniformly for all k ≤ n − 1. We show analogous results when the edge weights are drawn from an exponential distribution. The same results characterize the collectively cheapest k edge‐disjoint paths, that is, a minimum‐cost k‐flow. We also obtain the expectation of Xk conditioned on the existence of Pk.

Clustering sensor nodes is one of the effective ways which extends the life time of sensor networks. In this paper, a learning automata-based clustering algorithm (LACA) for a sensor network is proposed. The proposed algorithm is completely distributed and independent of network size and topology. The performance of the LACA via the computer simulation was evaluated and compared with other clustering algorithms. The simulation results show the high performance of the proposed clustering algorithm.

Segmentation task plays an important role in image processing. In this paper, we attempt to extract information from images using texture analysis. Moreover, we propose characterization of pixels in images to define the similarity relation between them. These are based on textural information and findings of shortest paths in the graph representation of images. To reflect effectiveness of our method, we apply it to the benchmark Berkeley image database and we compare it to well-established image segmentation methods (sum and difference histograms for texture classification method, Mean-Shift method and mixture of Gaussian distributions method). The proposed approach achieves the best segmentation results measured by distance-based metrics. The experimental results show that our approach is efficient method for texture analysis and image segmentation.

Stochastic Event Graph (SEG) is a useful tool to model some discrete event dynamic systems and evaluate their performance. In this paper, the evolution of the system is analysed by unfolded SEG and its state equations are presented. By using the tree structure of the correlative property of the state variables, an algorithm for the probability distributions of state variables is derived. Finally, for an example of decision making organization, the steady solutions are reached after two iterations. In comparison with Markov Chain analysis, this algorithm has better convergence, and can solve problems with general distributions of service time. It can obtain not only steady distributions, but also transient properties. >