Extended distributed learning automata

Many factors influence the connection states between nodes of wireless sensor networks, such as physical distance, and the network load, making the network’s edge length dynamic in abundant scenarios. This dynamic property makes the network essentially form a graph with stochastic edge lengths. In this paper, we study the stochastic shortest path problem on a directional graph with stochastic edge lengths, using reinforcement learning algorithms. we regard each edge length as a random variable following unknown probability distribution and aim to find the stochastic shortest path on this stochastic graph. We evaluate the performance of path-finding algorithms using regret, which represents the cumulative reward difference between the practical path-finding algorithm and the optimal strategy that chooses the global stochastic shortest path every time. We model the path-finding procedure as a Markov decision process and propose two online path-finding algorithms: Q SSP algorithm and SARSA SSP algorithm, both combined with specifically-devised average reward mechanism. We justify the convergence property and correctness of the proposed algorithms theoretically. Experiments conducted on two benchmark graphs illustrate the superior performance of the proposed Q SSP algorithm which outperforms the SARSA SSP algorithm and other competitive algorithms about the regret metric.

Given a sample from an unknown probability distribution over strings, there exist algorithms for inferring the structure and parameters of stochastic grammatical representations of the unknown distribution, i.e. string grammars. Despite the fact that research on grammatical representations of sets of graphs has been conducted since the late 1960’s, almost no work has considered the possibility of stochastic graph grammars and no algorithms exist for inferring stochastic graph grammars from data. This paper presents PEGG, an algorithm for estimating the parameters of stochastic context-free graph grammars given a sample from an unknown probability distribution over graphs. It is established formally that for a certain class of graph grammars PEGG finds parameter estimates in polynomial time that maximize the likelihood of the data, and preliminary empirical results demonstrate that the algorithm performs well in practice.KeywordsGraph TransformationGraph GrammarMaximum Likelihood ParameterStart SymbolEstimate Maximum Likelihood ParameterThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Sampling algorithms for stochastic graphs: A learning automata approach

We consider a balance-constrained stochastic bottleneck spanning tree problem (BCSBSTP) where edge weights are independently distributed but may follow arbitrary continuous distributions. The goal is to minimize a threshold variable that may be exceeded by the maximum edge weight at certain risk, subject to the minimum edge weight being no less than a fixed threshold with a probability guarantee. We characterize these two requirements as chance constraints, which are typically used for bounding the risk of undesirable random outcomes. Given independently distributed edge weights, we reformulate BCSBSTP as a mixed-integer nonlinear program, approximated by two mixed-integer linear programs based on special ordered set of type one (SOS1) and special ordered set of type two (SOS2) variables. By relaxing the probabilistic guarantee on the minimum edge weight in BCSBSTP, we also consider a stochastic bottleneck spanning tree problem (SBSTP), of which optimal tree solutions are approximated via a bisection algorithm in pseudopolynomial time. We demonstrate computational results of our models and algorithms by testing randomly generated instances with edge weights following a diverse set of independent distributions.

In this paper, we study the problem of finding the shortest path in stochastic graphs and propose an iterative algorithm for solving it. This algorithm is based on distributed learning automata (DLA), and its objective is to use a DLA for finding the shortest path from the given source node to the given destination node whose weight is minimal in expected sense. At each stage of this algorithm, DLA specifies edges needed to be sampled. We show that the given algorithm finds the shortest path with minimum expected weight in stochastic graphs with high probability which can be close to unity as much as possible. We compare the given algorithm with some distributed learning automata-based iterative algorithms, a particle swarm optimization-based algorithm, an ant colony-based algorithm, a Q-learning-based algorithm, and an actor–critic-based algorithm for finding the shortest path. Computer experiments show that the proposed algorithm requires fewer edge samples to find the shortest path than the previously introduced DLA-based algorithms.

The minimum connected dominating set (MCDS) of a given graph G is the smallest sub-graph of G such that every vertex in G belongs either to the sub-graph or is adjacent to a vertex of the sub-graph. Finding the MCDS in an arbitrary graph is a NP-Hard problem, and several approximation algorithms have been proposed for solving this problem in deterministic graphs, but to the best of our knowledge no work has been done on finding the MCDS in stochastic graphs. In this paper, the MCDS problem in the stochastic graphs is first introduced, and then a learning automata-based approximation algorithm called SCDS is proposed for solving this problem when the probability distribution function of the vertex weight is unknown. It is shown that by a proper choice of the parameters of the proposed algorithm, the probability with which the proposed algorithm find the MCDS is close enough to unity. The simulation results show the efficiency of the proposed algorithm in terms of the number of samplings.

An approximation algorithm is suggested for the problem of finding a d-regular spanning connected subgraph of maximum weight in a complete undirected weighted n-vertex graph. Probabilistic analysis of the algorithm is carried out for the problem with random input data (some weights of edges) in the case of a uniform distribution of the weights of edges and in the case of a minorized type distribution. It is shown that the algorithm finds an asymptotically optimal solution with time complexity O(n 2) when d = o(n). For the minimization version of the problem, an additional restriction on the dispersion of weights of the graph edges is added to the condition of the asymptotical optimality of the modified algorithm.

In Bayesian statistical inference, an unknown probability distribution is estimated from a sample using a statistical model and a prior. In general, a pair of a model and a prior may or may not be appropriate for unknown distribution; some evaluation procedures of the estimated result are necessary. If a statistical model is regular and the likelihood function can be approximated by some Gaussian function, then AIC and BIC can be applied to such evaluation processes. However, if a statistical model contains hierarchical structure or latent variables, then regularity condition is not satisfied. The information criteria WAIC and WBIC are devised so as to estimate the generalization loss and the free energy, respectively, even if the posterior distribution is far from any normal distribution and even if the unknown true distribution is not realizable by a statistical model. In this paper, we introduce mathematical foundation and computing methods of WAIC and WBIC in a normal mixture which is a typically singular statistical model, and discuss their properties in statistical inference. Also, we study the case that samples are not independently and identically distributed, for example, they are conditional independent or exchangeable.

Blocking flows were introduced by Dinic (Soviet Math Doklady 11: 1277–1280, 1970) to speed up the computation of maximum network flows. They have been used in algorithms for problems such as maximum cardinality matching of bipartite graphs Hopcroft and Karp (SIAM J Comput 2(4), 225–231, 1973) and general graphs Micali and Vazirani (in: Proceedings of the 21st Annual Symposium on Foundations of Computer Science, 17–27, 1980), maximum weight matching of general graphs Gabow and Tarjan (J ACM 38(4), 815–853, 1991), and many others. The blocking algorithm of Gabow and Tarjan (1991) for matching is based on depth-first search. We extend the depth-first search approach to find f-factors of general multigraphs. Here f is an arbitrary integral-valued function on vertices, an f-matching is a subgraph where every vertex x has degree $$\le f(x)$$ , an f-factor has equality in every degree bound. A set of blocking trails for an f-matching M is a maximal collection $$\mathcal{A}$$ of edge-disjoint augmenting trails such that $$M\bigoplus _{A\in {\mathcal{A}}} A $$ is a valid f-matching. Blocking trails are needed in efficient algorithms for maximum cardinality f-matching Huang and Pettie (Algorithmica 84(7): 1952–1992, 2022), maximum weight f-factors/matchings by scaling Duan et al. (In: Proceedings of the 47th International Colloquium on Automata, Languages, and Programming (ICALP 2020), Vol. 168 of LIPIcs, 41:1-41:17, 2020; Gabow (A weight-scaling algorithm for f-factors of multigraphs. arXiv:2010.01102 , 2020), and approximate maximum weight f-factors and f-edge covers Huang and Pettie (2022). Since these algorithms find many sets of blocking trails, the time to find blocking trails is a dominant factor in the running time. Our blocking trail algorithm runs in linear time O(m). In independent work and using a different approach, Huang and Pettie (2022) present a blocking trails algorithm using time $$O(m\alpha (m,n))$$ . As examples of the time bounds for the above applications, an approximate maximum weight f-factor is found in time $$O(m\,\alpha (m,n))$$ using Huang and Pettie (2022), and our algorithm eliminates the factor $$\alpha (m,n)$$ . Similarly a maximum weight f-factor is found in time $$O(\sqrt{\Phi \log \Phi }\, m\,\alpha (m,n)\, \log (\Phi W))\,$$ using Huang and Pettie (2022) , ( $$\Phi =\sum _{v\in V} f(v)$$ , W the maximum edge weight) and our algorithm eliminates the $$\alpha (m,n)$$ factor, making the time within a factor $$\sqrt{\log {\Phi }}$$ of the bound for bipartite multigraphs. The technical difficulty for this work stems from the fact that a fixed vertex can occur many times in a given search. This does not occur in ordinary matching or in algorithms for maximum cardinality or maximum weight f-matching. These multiple occurrences can create a new variant of blossom, the “skew blossom”. Also they can make blossoms become “incomplete”, i.e., partially processed yet still relevant in future searches.

We present an algorithm which calculates a minimum cut and its weight in an undirected graph with nonnegative real edge weights, n vertices and m edges, in time O(max(log n, min(m/n,δG/e)) n2), where e is the minimal edge weight, and δG is the minimal weighted degree. For integer edge weights this time is further improved to O(δG n2) and O(λG n2). In both cases these bounds are improvements of the previously known best bounds of deterministic algorithms. These were O(nm + n2 log n) for real edge weights and O(nM + n2) and O(M + λG n2) for integer weights, where M is the sum of all edge weights.

We present a new scaling algorithm for maximum (or minimum) weight perfect matching on general, edge weighted graphs. Our algorithm runs in [EQUATION] time, [EQUATION] per scale, which matches the running time of the best cardinality matching algorithms on sparse graphs [29, 18]. Here m, n, and N bound the number of edges, vertices, and magnitude of any integer edge weight. Our result improves on a 25-year old algorithm of Gabow and Tarjan, which runs in [EQUATION] log(nN)) time.

The petroleum industry uses high level dynamic simulations applied to geocellular models to guide forecasts of oil, gas and water production. Uncertainty in model choice and input variable selection is often addressed through large numbers of computationally slow Monte Carlo simulations designed around physics based models. Here, an alternate approach is proposed, which uses a relatively small amount of data and a reduced number of simulations of the high level physics model to train a fast (to evaluate) proxy or surrogate model based on a Polynomial Chaos Expansion. We give details of the theory and incorporated techniques, which significantly increase flexibility. Input variables (e.g. cell-by-cell variations in porosity and permeability) are sampled from unknown probability distributions and sensitivity analysis is based on low level proxy models. The theory is tested by developing proxy models to predict total gas production from a five-spot well configuration in the Hermitage area that taps into the Walloon Coal Measures of the Surat Basin in Queensland. Synthetic training data is simulated using commercial dynamic simulation software based on a high level physics model.

Frequent Itemsets (FIs) mining is a fundamental primitive in data mining. It requires to identify all itemsets appearing in at least a fraction θ of a transactional dataset D. Often though, the ultimate goal of mining D is not an analysis of the dataset per se, but the understanding of the underlying process that generated it. Specifically, in many applications D is a collection of samples obtained from an unknown probability distribution π on transactions, and by extracting the FIs in D one attempts to infer itemsets that are frequently (i.e., with probability at least θ) generated by π, which we call the True Frequent Itemsets (TFIs). Due to the inherently stochastic nature of the generative process, the set of FIs is only a rough approximation of the set of TFIs, as it often contains a huge number of false positives, i.e., spurious itemsets that are not among the TFIs. In this work we design and analyze an algorithm to identify a threshold such that the collection of itemsets with frequency at least in contains only TFIs with probability at least 1 – δ, for some user-specified δ. Our method uses results from statistical learning theory involving the (empirical) VC-dimension of the problem at hand. This allows us to identify almost all the TFIs without including any false positive. We also experimentally compare our method with the direct mining of at frequency θ and with techniques based on widely-used standard bounds (i.e., the Chernoff bounds) of the binomial distribution, and show that our algorithm outperforms these methods and achieves even better results than what is guaranteed by the theoretical analysis.

This article focuses on the stability issue of switched network control systems (SNCSs) under deception attacks described by a Bernoulli process with unknown probability distribution. The false information in deception attacks is unknown but bounded and may be state dependent or state independent. By means of the input-to-state stability (ISS) tool and the convex combination method, an improved lemma is first developed for SNCSs, which facilitates the derivations of our results. After that, some attack-independent sufficient conditions for the ISS of SNCSs are obtained for mode-dependent average dwell time switching and stochastic switching, respectively. Different from existing results, the concerned switching contributes to the stability of SNCSs, which benefits the ISS performance of SNCSs even though the unknown deception attacks cause all subsystems to be non-ISS. The proposed results provide an effective solution with strong robustness to deal with unknown deception attacks or denial-of-service attacks.

We present a sequential framework for change detection. This framework allows us to use multiple images from reference and mission passes of a scene of interest in order to improve detection performance. It includes a change statistic that is easily updated when additional data becomes available. Detection performance using this statistic is predictable when the reference and image data are drawn from known distributions. We verify our performance prediction by simulation. Additionally, we show that detection performance improves with additional measurements on a set of synthetic aperture radar images and a set of visible images with unknown probability distributions.