Modeling graphs using dot product representations

Graph representation learning is a fundamental technique for machine learning (ML) on complex networks. Given an input network, these methods represent the vertices by low-dimensional real-valued vectors. These vectors can be used for a multitude of downstream ML tasks. We study one of the most important such task, link prediction. Much of the recent literature on graph representation learning has shown remarkable success in link prediction. On closer investigation, we observe that the performance is measured by the AUC (area under the curve), which suffers biases. Since the ground truth in link prediction is sparse, we design a vertex-centric measure of performance, called the VCMPR@k plots. Under this measure, we show that link predictors using graph representations show poor scores. Despite having extremely high AUC scores, the predictors miss much of the ground truth. We identify a mathematical connection between this performance, the sparsity of the ground truth, and the low-dimensional geometry of the node embeddings. Under a formal theoretical framework, we prove that low-dimensional vectors cannot capture sparse ground truth using dot product similarities (the standard practice in the literature). Our results call into question existing results on link prediction and pose a significant scientific challenge for graph representation learning. The VCMPR plots identify specific scientific challenges for link prediction using low-dimensional node embeddings.

Let [Formula: see text] be an integer. From a set of [Formula: see text]-dimensional vectors, we obtain a [Formula: see text]-dot product graph by letting each vector [Formula: see text] correspond to a vertex [Formula: see text] and by adding an edge between two vertices [Formula: see text] and [Formula: see text] if and only if their dot product [Formula: see text], for some fixed, positive threshold [Formula: see text]. Dot product graphs can be used to model social networks. Recognizing a [Formula: see text]-dot product graph is known to be [Formula: see text]-hard for all fixed [Formula: see text]. To understand the position of [Formula: see text]-dot product graphs in the landscape of graph classes, we consider the case [Formula: see text], and investigate how [Formula: see text]-dot product graphs relate to a number of other known graph classes including a number of well-known classes of intersection graphs.

Because of employing elevation adaptivity, three-dimensional space-time adaptive processing (3D STAP) can achieve better performance than conventional two-dimensional (2D) STAP at the cost of higher computational load and sample support requirement. To overcome these shortcomings, a tri-iterative 3D STAP method based on correlation matrix for airborne radar is developed. In the proposed method, the quadratic cost function used in the optimum STAP is converted into three quadratic functions by using submatrices of the space-time correlation matrix, and the full dimension weight vector can be separated into three lower dimensional weight vectors. By iteratively optimizing these lower dimensional weight vectors, the proposed method can significantly decrease the computational load and training samples requirement. Experiment results using both simulated data and measured radar data demonstrate the effectiveness of the proposed method.

The task of link prediction for knowledge graphs is to predict missing relationships between entities. Knowledge graph embedding, which aims to represent entities and relations of a knowledge graph as low dimensional vectors in a continuous vector space, has achieved promising predictive performance. If an embedding model can cover different types of connectivity patterns and mapping properties of relations as many as possible, it will potentially bring more benefits for link prediction tasks. In this paper, we propose a novel embedding model, namely LineaRE, which is capable of modeling four connectivity patterns (i.e., symmetry, antisymmetry, inversion, and composition) and four mapping properties (i.e., one-to-one, one-to-many, many-to-one, and many-to-many) of relations. Specifically, we regard knowledge graph embedding as a simple linear regression task, where a relation is modeled as a linear function of two low-dimensional vector-presented entities with two weight vectors and a bias vector. Since the vectors are defined in a real number space and the scoring function of the model is linear, our model is simple and scalable to large knowledge graphs. Experimental results on multiple widely used real-world datasets show that the proposed LineaRE model significantly outperforms existing state-of-the-art models for link prediction tasks.

A method is proposed to obtain a model of a dynamic system with a state vector of high dimension. The model is derived by aggregating the original system state vector into a lower-dimensional vector. Some properties of the aggregation method are investigated in the paper. The concept of aggregation, a generalization of that of projection, is related to that of state vector partition and is useful not only in building a model of reduced dimension, but also in unifying several topics in the control theory such as regulators with incomplete state feedback, characteristic value computations, model controls, and bounds on the solution of the matrix Riccati equations, etc. Using the quantitative definition of weak coupling proposed by Milne, a suboptimal control policy for the weakly coupled system is derived. Questions of performance degradation and of stability of such suboptimally controlled systems are also answered in the paper.

This paper presents algorithms to deal with problems associated with indexing high-dimensional feature vectors, which characterize video data. Indexing high-dimensional vectors is well known to be computationally expensive. Our solution is to optimally split the high dimensional vector into a few low dimensional feature vectors and querying the system for each feature vector. This involves solving an important subproblem: developing a model of retrieval which enables us to query the system efficiently. Once we formulate the retrieval problem in terms of a retrieval model, we present an optimality criterion to maximize the number of results using this model. The criterion is based on a novel idea of using the underlying probability distribution of the feature vectors. A branch-and-prune strategy optimized per each query, is developed. This uses the set of features derived from the optimality criterion. Our results show that the algorithm performs well, giving a speedup of a factor of 25 with respect to a linear search, while retaining the same level of recall.

Generalized incidence theorems, homogeneous forms and sum–product estimates in finite fields

The newest airborne and spaceborne imaging sensors, unlike their predecessors, generate data at tens to hundreds of wavelengths simultaneously. The advent of these imaging spectrometers greatly increases the detail of the information available to the analyst, but the concomitant disadvantages of vastly increased data volume and complexity have stimulated the search for more efficient and accurate hyperspectral data exploitation techniques. This paper describes two neural network models used in conjunction to analyze hyper-spectral image data. The first neural network is based on an unsupervised backward error propagation model. This network generates without supervision low dimensional discriminant feature vectors from the original high dimensional signals. These features are analyzed by the second neural network, which is a layered feed-forward neural network based on statistical principles, namely Parzen window density estimation and Bayes decision rule. This network estimates class-conditional probability density functions using training samples and classifies data points so that the expected risk is minimized. Results of applying these two neural networks to 210-band AVIRIS data are presented.

This paper proposes a hybrid method for face recognition using local features and statistical feature extraction methods. First, a dense set of local feature points are extracted in order to represent a facial image. Each local feature point is described by the keypoint descriptor defined by SIFT feature. Then, the statistical feature extraction methods, PCA and LDA, are applied to the set of local feature descriptors in order to find low dimensional features. With the obtained low dimensional feature vectors, we can conduct face recognition task efficiently using a simple classifier. Through computational experiments on benchmark data sets, we show that the proposed method is superior to the conventional PCA and LDA in the classification performance. In addition, we also show that the proposed method can achieve remarkable improvement in the processing time compared to the conventional keypoint matching methods proposed for local features.KeywordsFace RecognitionLocal FeatureSift FeatureGray Level IntensityConventional Statistical MethodThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

On Image Forgery Detection Using Two Dimensional Discrete Cosine Transform and Statistical Moments

Successful classification depends on the selection of the distinctive features and the effective channel subset used in the classification. In this study, novel and practical methods are proposed for determining the distinctive features and detecting effective channel subsets in the multi channel classification systems such as EEG. Two different feature extraction methods are compared in the study. The first one is based on classical Wavelet transform and the second is our proposed approach which used the slope of signal segments. Feature vectors are generated from some signal properties such as the mean, standard deviation, numerical integral of the Wavelet coefficients for classical Wavelet transform based feature extraction method. For our proposed method, only the slopes of signal segments are used for the feature vectors. In the proposed Signal Path Slope (SPS) feature extraction method, differently from the classical Wavelet based method, a Savitzky Golay (S-G) filter with an optimal frame length is applied to the signal before segmentation to make the path of the signal more prominent in time domain. In this way, the distinctive classification features are extracted by using S-G filter. For channel selection, an iterative channel selection method based on the classification results which divide the dataset labelled dataset into two groups as % 90 pre-training and %10 pre-test data is proposed. The dataset provided as dataset-3 in BCI competition IV is used in this study. The feature vectors extracted by using the proposed methods are classified for each method with the Support Vector Machine classifier. The results are given comparatively and it is observed that our proposed method has less computational complexity and more successful classification than Wavelet based classical feature extraction methods. The highest classification accuracies of % 67.74 and % 49.27 for subject-1 and subject-2 respectively are obtained with a low dimensional feature vector by proposed SPS feature extraction method. The classification accuracies achieved in the study are increased by % 8.24 for subject-1 and % 14.97 for subject-2 when compared average of the competition results. The significant increase in the success for both subjects shows the consistency of the proposed methods. By this study, it is observed that there is a subject-specific signal pattern related to motor imagery tasks in the brain. This pattern distinctive features is successfully determined by using the proposed methods.

Mammography is an important diagnostic tool for early detection of breast cancer. Masses in mammogram are characterized by their shape and margin of their contours. A typical contour of benign mass is round and oval, whereas a malignant mass has irregular and spiculated. Many studies in literature have shown that features extracted from 1D signature of 2D mass contour can distinguish benign and malignant lesion. The features extracted from 1D signature results in low dimensional feature vector and in turn doesn't require feature selection step for classification. The proposed research study applies a Beam angle statistics (BAS) to convert 2D contour into 1D signature. Further, the features are extracted from the 1D BAS signatures in order to classify masses. K-nearest neighbor (KNN), support vector machines (SVM), Adaboost decision tree and artificial neural networks (ANN) are employed to categorize breast masses into benign and malignant using features of BAS signature. The proposed methodology was tested on 147 contours, obtaining accuracy of 88.8% accuracy. The proposed system was also compared with recent literature described related breast cancer analysis.

This paper presents an approach to full-body human pose recognition. Inputs to the proposed approach are pairs of silhouette images obtained from wide baseline binocular cameras. Through multilinear analysis, low dimensional view-invariant pose coefficient vectors can be extracted from these stereo silhouette pairs. Taking these pose coefficient vectors as features, the Universum method is trained and used for pose recognition. Experiment results obtained using real image data showed the efficacy of the proposed approach.

The matrix factorization recommendation algorithm does not consider characteristics of the recommendation object itself, resulting in poor recommendation results. Therefore, a matrix factorization recommendation algorithm based on knowledge map representation learning is proposed. Firstly, the recommendation object is represented as a low dimensional semantic vector by using the knowledge map distributed representation learning algorithm. Then the semantic similarity between objects is calculated, and the semantic similarity is incorporated into the objective optimization function of matrix factorization, so that the feature vectors obtained by matrix factorization can also contain semantic knowledge, which makes up for the shortcoming of matrix factorization recommendation algorithm that does not consider characteristics of the recommendation object itself from the semantic perspective. The experimental results show that the improved algorithm has higher accuracy, recall and coverage than the traditional matrix factorization recommendation algorithm.

The representation learning in knowledge graph aims to embed the semantic information of the research object into dense low dimensional distributed real valued vectors. The classical knowledge representation model, TransR, uses the relation specific matrix to represent the entities and relations in different semantic spaces, so as to solve the separation problem of semantic space. However, the TransR model only considers the different types of relations, ignores the entity types, and has defects in the processing of inverse relations. To solve the model problems, a novel knowledge representation method for multiple pattern embeddings based on entity-relation mapping matrix (M-TransER) is presented. The M-TransER model introduces the entity-relation mapping matrix, which makes the model consider the entity type while considering the relation type. On the basis that entities and relations are modeled as positive translation geometric distance, the inverse translation geometric distance model and the symmetrical relation model are fused. Eliminate the defects of the model in dealing with inverse relations, and improve the efficiency of symmetric relation processing. Experimental results on open datasets show that the M-TransER model has made great progress and demonstrated its superiority over the existing models and methods in both link prediction and triple classification tasks.