Possibilistic Conditional Preference Networks

Possibilistic preference networks

This thesis concerns graphical models, mainly with discrete variables, a class of probabilistic models that can be understood pictorially using an interaction graph. Some classical graphical models includes the naive Bayes classifier and the Ising model, where the interaction graph of the former has directed edges, and the latter has undirected edges. At the core of graphical models is the concept of conditional independence; X is conditionally independent of Y given variable Z if by knowing Z, Y is irrelevant for X. Having access to such knowledge typically simplifies the model and hence the interaction graph. For classification problems, the structure of the interaction graph may be of lesser importance, whereas the structure may be essential in problems where the goal is the understand the underlying phenomenon. A large number of different graphical models exists with special structures of the interaction; graphs with both directed and undirected edges, both discrete and continuous variables etc. Common to all these is that the graphical structure must be learned either from data, also referred to as structure learning, or specified through expert knowledge. Given an interaction graph, calculating posterior probabilities, also known as inference, is essential in graphical models. Although both structure learning and inference are, in general, NP-complete problems there exist efficient heuristics for both, and even exact methods for inference. Most notably is the junction tree algorithm.

Adjusting for Selection Effects in Epidemiologic Studies

In this article we show the duality between tensor networks and undirected graphical models with discrete variables. We study tensor networks on hypergraphs, which we call tensor hypernetworks. We show that the tensor hypernetwork on a hypergraph exactly corresponds to the graphical model given by the dual hypergraph. We translate various notions under duality. For example, marginalization in a graphical model is dual to contraction in the tensor network. Algorithms also translate under duality. We show that belief propagation corresponds to a known algorithm for tensor network contraction. This article is a reminder that the research areas of graphical models and tensor networks can benefit from interaction.

BackgroundMixed graphical models (MGMs) are graphical models learned over a combination of continuous and discrete variables. Mixed variable types are common in biomedical datasets. MGMs consist of a parameterized joint probability density, which implies a network structure over these heterogeneous variables. The network structure reveals direct associations between the variables and the joint probability density allows one to ask arbitrary probabilistic questions on the data. This information can be used for feature selection, classification and other important tasks.ResultsWe studied the properties of MGM learning and applications of MGMs to high-dimensional data (biological and simulated). Our results show that MGMs reliably uncover the underlying graph structure, and when used for classification, their performance is comparable to popular discriminative methods (lasso regression and support vector machines). We also show that imposing separate sparsity penalties for edges connecting different types of variables significantly improves edge recovery performance. To choose these sparsity parameters, we propose a new efficient model selection method, named Stable Edge-specific Penalty Selection (StEPS). StEPS is an expansion of an earlier method, StARS, to mixed variable types. In terms of edge recovery, StEPS selected MGMs outperform those models selected using standard techniques, including AIC, BIC and cross-validation. In addition, we use a heuristic search that is linear in size of the sparsity value search space as opposed to the cubic grid search required by other model selection methods. We applied our method to clinical and mRNA expression data from the Lung Genomics Research Consortium (LGRC) and the learned MGM correctly recovered connections between the diagnosis of obstructive or interstitial lung disease, two diagnostic breathing tests, and cigarette smoking history. Our model also suggested biologically relevant mRNA markers that are linked to these three clinical variables.ConclusionsMGMs are able to accurately recover dependencies between sets of continuous and discrete variables in both simulated and biomedical datasets. Separation of sparsity penalties by edge type is essential for accurate network edge recovery. Furthermore, our stability based method for model selection determines sparsity parameters faster and more accurately (in terms of edge recovery) than other model selection methods. With the ongoing availability of comprehensive clinical and biomedical datasets, MGMs are expected to become a valuable tool for investigating disease mechanisms and answering an array of critical healthcare questions.

In this dissertation, we consider models with low-rank and group-sparse components. First, we investigate robust principal component analysis, where the low-rank component represents the principal components, and the group-sparse component accounts for corruptions in the data. We propose a model for the general setting, where groups of observed variables can be corrupted. Second, we generalize fused latent and graphical models to the class of conditional Gaussian distributions with mixed observed discrete and quantitative variables. Fused latent and graphical models are characterized by a decomposition of the pairwise interaction parameter matrix into a group-sparse component of direct interactions and a low-rank component of indirect interactions due to a small number of quantitative latent variables. All models in this thesis can be learned by solving convex optimization problems with low-rank and group-sparsity inducing regularization terms. For fused latent and graphical models, there is an additional likelihood term. We show that under identifiability assumptions, a given true model can be recovered exactly (principal component analysis) or consistently (fused latent and graphical models, high-dimensional setting) by solving the respective optimization problems. We also present heuristics for selecting the regularization parameters that appear in the optimization problems. We conduct experiments on synthetic and real-world data to support our theory.

BackgroundGait kinematics measured during equine gait analysis are typically evaluated by analysing (asymmetry‐based) discrete variables (eg, peak values) obtained from continuous kinematic signals (eg, timeseries of datapoints). However, when used for the assessment of complex cases of lameness, such as bilateral lameness, discrete variable analysis might overlook relevant functional adaptations.ObjectivesThe overall aim of this paper is to compare continuous and discrete data analysis techniques to evaluate kinematic gait adaptations to lameness.Study designMethod comparison.MethodsSixteen healthy Shetland ponies, enrolled in a research programme in which osteochondral defects were created on the medial trochlear ridges of both femurs, were used in this study. Kinematic data were collected at trot on a treadmill before and at 3 and 6 months after surgical intervention. Statistical parametric mapping and linear mixed models were used to compare kinematic variables between and within timepoints.ResultsBoth continuous and discrete data analyses identified changes in pelvis and forelimb kinematics. Discrete data analyses showed significant changes in hindlimb and back kinematics, where such differences were not found to be significant by continuous data analysis. In contrast, continuous data analysis provided additional information on the timing and duration of the differences found.Main limitationsA limited number of ponies were included.ConclusionsThe use of continuous data provides additional information regarding gait adaptations to bilateral lameness that is complementary to the analysis of discrete variables. The main advantage lies in the additional information regarding time dependence and duration of adaptations, which offers the opportunity to identify functional adaptations during all phases of the stride cycle, not just the events related to peak values.

Graph rewriting for a partial ordering semantics of concurrent constraint programming

Associations between multiple discrete measures are often due to collapsing over other variables. When the variables collapsed over are unobserved and continuous, log-multiplicative association models, including log-linear models with linear-by-linear interactions for ordinal categorical data and extensions of Goodman's (1979, 1985) RC(M) association model for multiple nominal and/or ordinal categorical variables, can be used to study the relationship between the observed discrete variables and the unobserved continuous ones, and to study the unobserved variables. The derivation and use of log-multiplicative association models as latent variable models for discrete variables are presented in this paper. The models are based on graphical models for discrete and continuous variables where the variables follow a conditional Gaussian distribution. The models have many desirable properties, including having schematic or graphical representations of the system of observed and unobserved variables, the log-multiplicative models can be read from the graphs, and estimates of the means, variances, and covariances of the latent variables given values on the observed variables are a function of the log-multiplicative model parameters. To illustrate some of the advantageous aspects of these models, two examples are presented. In one example, responses to items from the General Social Survey (Davis and Smith 1996) are modeled, and in the other example, panel data from two groups (Coleman 1964) are analyzed.

Stable graphical model estimation with Random Forests for discrete, continuous, and mixed variables

Mathematically, a Bayesian graphical model is a representation of the joint probability distribution for a set of variables. The most frequently used type of Bayesian graphical models are Bayesian networks, but other types exist and Bayesian networks can be seen as a special type of Chain Graphs. The structural part of a Bayesian graphical model is a graph consisting of nodes and edges. The nodes represent variables. The set of all variables, called the universe, is denoted U . A variable may be discrete or continuous. For the sake of simplicity this article focuses on models with only discrete variables, and refers to the variable values as states. An edge between two nodes A and B indicates a direct influence between the state of A and the state of B .

Conjunctive Bayesian networks (CBNs) are graphical models that describe the accumulation of events which are constrained in the order of their occurrence. A CBN is given by a partial order on a (finite) set of events. CBNs generalize the oncogenetic tree models of Desper et al. by allowing the occurrence of an event to depend on more than one predecessor event. The present paper studies the statistical and algebraic properties of CBNs. We determine the maximum likelihood parameters and present a combinatorial solution to the model selection problem. Our method performs well on two datasets where the events are HIV mutations associated with drug resistance. Concluding with a study of the algebraic properties of CBNs, we show that CBNs are toric varieties after a coordinate transformation and that their ideals possess a quadratic Gröbner basis.

Conditional preference networks (CP-nets) are used as intuitive graphical tools to represent conditional preference statements regarding the values of a set of attributes. Making use of CP-nets to solve certain learning problems has attracted growing attention from scholars and become a hot topic in the field of artificial intelligence. Therefore, many methods have been proposed to solve these learning problems. However, these approaches suffer from two main disadvantages: the amount of time required and their lack in concrete structures. To overcome these limitations, in this paper, we first provide theoretical support for the use of a conditional independent test for learning the structure of CP-nets. Second, we propose the dependent degree to calculate the dependency relationship among attributes. Finally, we present an algorithm to obtain the structures of CP-nets. Beyond that, a number of database samples have been reduced by filtering out insignificant or noise data, and a concrete structure of learned CP-nets with 18 attributes is given. The experiments show that our approach can obtain a better structure of CP-nets without materially increasing the time required for the process and put forward contrast to methods presented antecedently.

We consider the problem of the estimation of a high-dimensional probability\ndistribution from i.i.d. samples of the distribution using model classes of\nfunctions in tree-based tensor formats, a particular case of tensor networks\nassociated with a dimension partition tree. The distribution is assumed to\nadmit a density with respect to a product measure, possibly discrete for\nhandling the case of discrete random variables.\n After discussing the representation of classical model classes in tree-based\ntensor formats, we present learning algorithms based on empirical risk\nminimization using a $L^2$ contrast.\n These algorithms exploit the multilinear parametrization of the formats to\nrecast the nonlinear minimization problem into a sequence of empirical risk\nminimization problems with linear models. A suitable parametrization of the\ntensor in tree-based tensor format allows to obtain a linear model with\northogonal bases, so that each problem admits an explicit expression of the\nsolution and cross-validation risk estimates. These estimations of the risk\nenable the model selection, for instance when exploiting sparsity in the\ncoefficients of the representation.\n A strategy for the adaptation of the tensor format (dimension tree and\ntree-based ranks) is provided, which allows to discover and exploit some\nspecific structures of high-dimensional probability distributions such as\nindependence or conditional independence.\n We illustrate the performances of the proposed algorithms for the\napproximation of classical probabilistic models (such as Gaussian distribution,\ngraphical models, Markov chain).\n

Graphical modelling is an important tool for the efficient representation and analysis of uncertain information in knowledge-based systems. While Bayesian networks and Markov networks from probabilistic graphical modelling are well-known for a couple of years, the field of possibilistic graphical modelling occurs as a new promising area of research. Possibilistic networks provide an alternative approach compared to probabilistic networks, whenever it is necessary to model uncertainty and imprecision as two different kinds of imperfect information. Imprecision in the sense of set-valued data has often to be considered in situations where data are obtained from human observations or non-precise measurement units. In this contribution we present a comparison of the background and perspectives of probabilistic and possibilistic graphical models, and give an overview on the current state of the art of possibilistic networks with respect to propagation and learning algorithms, applicable to data mining and data fusion problems.