Bayesian Analysis of Exponential Random Graph Models Using Stochastic Gradient Markov Chain Monte Carlo.

Deep network embedding with dimension selection

Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.

Advances in exponential random graph ( p*) models applied to a large social network

Deep learning models are traditionally used in big data scenarios. When there is not enough training data to fit a large model, transfer learning re-purpose the learned features from an existing model and re-train the lower layers for the new task. Bayesian inference techniques can be used to capture the uncertainty of the new model but it comes with a high computational cost. In this paper, the run time performance of an Stochastic Gradient Markov Chain Monte Carlo method using two different architectures is compared, namely GPU and multi-core CPU. As opposed to the widely usage of GPUs for deep learning, significant advantages from using modern CPU architectures.

We consider the problem of inferring an unknown number of clusters in multinomial count data, by estimating finite mixtures of multinomial distributions with or without covariates. Both Maximum Likelihood (ML) as well as Bayesian estimation are taken into account. Under a Maximum Likelihood approach, we provide an Expectation–Maximization (EM) algorithm which exploits a careful initialization procedure combined with a ridge-stabilized implementation of the Newton-Raphson method in the M-step. Under a Bayesian setup, a stochastic gradient Markov chain Monte Carlo (MCMC) algorithm embedded within a prior parallel tempering scheme is devised. The number of clusters is selected according to the Integrated Completed Likelihood criterion in the ML approach and estimating the number of non-empty components in overfitting mixture models in the Bayesian case. Our method is illustrated in simulated data and applied to two real datasets. The proposed methods are implemented in a contributed package, available online.

In this paper, a unified Bayesian max-margin discriminant projection framework is proposed, which is able to jointly learn the discriminant feature space and the max-margin classifier with different relationships between the latent representations and observations. We assume that the latent representation follows a normal distribution whose sufficient statistics are functions of the observations. The function can be flexibly realized through either shallow or deep structures. The shallow structure includes linear, nonlinear kernel-based functions, and even the convolutional projection, which can be further trained layerwisely to build a multilayered convolutional feature learning model. To take the advantage of the deep neural networks, especially their highly expressive ability and efficient parameter learning, we integrate Bayesian modeling and the popular neural networks, for example, mltilayer perceptron and convolutional neural network, to build an end-to-end Bayesian deep discriminant projection under the proposed framework, which degenerated into the existing shallow linear or convolutional projection with the single-layer structure. Moreover, efficient scalable inferences for the realizations with different functions are derived to handle large-scale data via a stochastic gradient Markov chain Monte Carlo. Finally, we demonstrate the effectiveness and efficiency of the proposed models by the experiments on real-world data, including four image benchmarks (MNIST, CIFAR-10, STL-10, and SVHN) and one measured radar high-resolution range profile dataset, with the detailed analysis about the parameters and computational complexity.

One way to avoid overfitting in machine learning is to use model parameters distributed according to a Bayesian posterior given the data, rather than the maximum likelihood estimator. Stochastic gradient Langevin dynamics (SGLD) is one algorithm to approximate such Bayesian posteriors for large models and datasets. SGLD is a standard stochastic gradient descent to which is added a controlled amount of noise, specifically scaled so that the parameter converges in law to the posterior distribution [WT11, TTV16]. The posterior predictive distribution can be approximated by an ensemble of samples from the trajectory.

Exponential random graph models (ERGMs) are increasingly applied to observed network data and are central to understanding social structure and network processes. The chapters in this edited volume provide a self-contained, exhaustive account of the theoretical and methodological underpinnings of ERGMs, including models for univariate, multivariate, bipartite, longitudinal and social-influence type ERGMs. Each method is applied in individual case studies illustrating how social science theories may be examined empirically using ERGMs. The authors supply the reader with sufficient detail to specify ERGMs, fit them to data with any of the available software packages and interpret the results.

Exponential random graph models have been widely used in social network analysis. However, these models are extremely difficult to handle from a statistical viewpoint, because of the intractable normalizing constant and model degeneracy. In this paper, we consider a fully Bayesian analysis for exponential random graph models using the adaptive exchange sampler, which solves the intractable normalizing constant and model degeneracy issues encountered in Markov chain Monte Carlo (MCMC) simulations. The adaptive exchange sampler can be viewed as a MCMC extension of the exchange algorithm, and it generates auxiliary networks via an importance sampling procedure from an auxiliary Markov chain running in parallel. The convergence of this algorithm is established under mild conditions. The adaptive exchange sampler is illustrated using a few social networks, including the Florentine business network, molecule synthetic network, and dolphins network. The results indicate that the adaptive exchange algorithm can produce more accurate estimates than approximate exchange algorithms, while maintaining the same computational efficiency.

In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. Recent work [Xu and Raginsky (2017)] has established a bound on the generalization error of empirical risk minimization based on the mutual information $I$ ( $S$ ; W) between the algorithm input $S$ and the algorithm output W, when the loss function is sub-Gaussian. We leverage these results to derive generalization error bounds for a broad class of iterative algorithms that are characterized by bounded, noisy updates with Markovian structure. Our bounds are very general and are applicable to numerous settings of interest, including stochastic gradient Langevin dynamics (SGLD) and variants of the stochastic gradient Hamiltonian Monte Carlo (SGHMC) algorithm. Furthermore, our error bounds hold for any output function computed over the path of iterates, including the last iterate of the algorithm or the average of subsets of iterates, and also allow for non-uniform sampling of data in successive updates of the algorithm.

Social network analysis is a suite of approaches for exploring relational data. Two approaches commonly used to analyze animal social network data are permutation-based tests of significance and exponential random graph models. However, the performance of these approaches when analyzing different types of network data has not been simultaneously evaluated. Here we test both approaches to determine their performance when analyzing a range of biologically realistic simulated animal social networks. We examined the false positive and false negative error rate of an effect of a two-level explanatory variable (e.g., sex) on the number and combined strength of an individual’s network connections. We measured error rates for two types of simulated data collection methods in a range of network structures, and with/without a confounding effect and missing observations. Both methods performed consistently well in networks of dyadic interactions, and worse on networks constructed using observations of individuals in groups. Exponential random graph models had a marginally lower rate of false positives than permutations in most cases. Phenotypic assortativity had a large influence on the false positive rate, and a smaller effect on the false negative rate for both methods in all network types. Aspects of within- and between-group network structure influenced error rates, but not to the same extent. In "grouping event-based" networks, increased sampling effort marginally decreased rates of false negatives, but increased rates of false positives for both analysis methods. These results provide guidelines for biologists analyzing and interpreting their own network data using these methods.

Since the 1970s, communication researchers have utilized social network analysis to understand mass, health, organizational, and interpersonal communication. This article introduces communication researchers to a new class of social network analysis methods, exponential random graph (p*) models. This new method represents the latest advancement in social network methodology and will enhance the trajectory of social network research in the communication discipline. The benefits of this class of models include allowing for the simultaneous estimation of attribute and structural parameters, accounting for the interdependent nature of network data, and retaining the complexity of network observations throughout the analysis. An example analysis using data from Shumate, Fulk, and Monge (2005) is provided to illustrate the potentials of exponential random graph modeling. Five different social network software programs capable of the analysis discussed in this article are introduced with regard to their respective benefits. Finally, a brief tutorial based on data from Palazzolo (2005) is given on how to conduct an ERGM analysis using the PNET software program.

Due to the development of the Internet and social networks, civil participation in political processes is taking new forms, not accounted for in the classical theoretical frameworks. In this study, we present a new methodology for researching these new, unconventional forms of civil participation. We use data collected from a social network site VK.com. Social network analysis methods, such as multilevel exponential random graph models (ERGM) were used to analyze the data. First proposed by Wang in 2013, multilevel ERGM allows us to model tie formation in a wide class of social networks. This method allows us to efficiently model group membership based on node attributes. Since the collected data is fundamentally 2-mode, this model allows us to identify the important factors that lead people to join online protest communities.

Social network analysis has typically concerned analysis of one type of tie connecting nodes of the same type. It has however been recognised that people are connected through multiple types of ties and that people in addition are affiliated with multiple types of non-people nodes. Exponential random graph models (ERGM) is a family of statistical models for social networks that at this point allows for a number of different types of network data, including one-mode networks, bipartite networks, multiplex data, as well as multilevel network data. Multilevel networks have been proposed as a joint representation of associations between multiple types of entities or nodes, such as people and organization, where two types of nodes gives rise to three distinct types of ties. The typical roster data collection method may be impractical or infeasible when the node sets are hard to detect or define or because of the cognitive demands on respondents. Multilevel multilayered networks allow us to consider a multitude of different sources of data and to sample on different types of nodes and relations. We consider modelling multilevel multilayered networks using exponential random graph models and extend a recently developed Bayesian data-augmentation scheme to allow for partially missing data. We illustrate the proposed inference procedures for the case of multilevel snowball sampling and sampling with error based on the Noordin Top network.

The United States has been experiencing an unprecedented level of opioid overdose-related mortality due in part to excessive use of prescription opioids. Peer-driven network interventions may be beneficial. A key assumption of social network interventions is that of some members of the network act as key players and can influence the behavior of others in the network. We used opioid prescription records to create a social network of patients who use prescription opioid in the state of Rhode Island. The study population was restricted to patients on stable opioid regimens who used one source of payment and received the same opioid medication from ≥ 3 prescribers and pharmacies. An exponential random graph model (ERGM) was employed to examine the relationship between patient attributes and the likelihood of tie formation and modularity was used to assess for homophily (the tendency of individuals to associate with similar people). We used multivariable logistic regression to assess predictors of high betweenness centrality, a measure of influence within the network. 372 patients were included in the analysis; average age was 51 years; 53% were female; 57% were prescribed oxycodone, 34% were prescribed hydrocodone and 9% were prescribed buprenorphine/naloxone. After controlling for the main effects in the ERGM model, homophily was associated with age group, method of payment, number and type of opioid prescriptions filled, mean daily dose, and number of providers seen. Type of opioid and number of prescribers were identified as significant predictors of high betweenness centrality. We conclude that patients who use multiple prescribers or have a diagnosis of opioid use disorder may help promote positive health behaviors or disrupt harmful behaviors in an opioid prescription network.