The general framework for few-shot learning by kernel HyperNetworks

Network analysis has been widely applied in many real-world tasks, such as gene analysis and targeted marketing. To extract effective features for these analysis tasks, network embedding automatically learns a low-dimensional vector representation for each node, such that the meaningful topological proximity is well preserved. While the embedding algorithms on pure topological structure have attracted considerable attention, in practice, nodes are often abundantly accompanied with other types of meaningful information, such as node attributes, second-order proximity, and link directionality. A general framework for incorporating the heterogeneous information into network embedding could be potentially helpful in learning better vector representations. However, it remains a challenging task to jointly embed the geometrical structure and a distinct type of information due to the heterogeneity. In addition, the real-world networks often contain a large number of nodes, which put demands on the scalability of the embedding algorithms. To bridge the gap, in this article, we propose a general embedding framework named Heterogeneous Information Learning in Large-scale networks (HILL) to accelerate the joint learning. It enables the simultaneous node proximity assessing process to be done in a distributed manner by decomposing the complex modeling and optimization into many simple and independent sub-problems. We validate the significant correlation between the heterogeneous information and topological structure, and illustrate the generalizability of HILL by applying it to perform attributed network embedding and second-order proximity learning. A variation is proposed for link directionality modeling. Experimental results on real-world networks demonstrate the effectiveness and efficiency of HILL.

Graph Neural Networks (GNNs) are de facto solutions to structural data learning. However, it is susceptible to low-quality and unreliable structure, which has been a norm rather than an exception in real-world graphs. Existing graph structure learning (GSL) frameworks still lack robustness and interpretability. This paper proposes a general GSL framework, SE-GSL, through structural entropy and the graph hierarchy abstracted in the encoding tree. Particularly, we exploit the one-dimensional structural entropy to maximize embedded information content when auxiliary neighbourhood attributes are fused to enhance the original graph. A new scheme of constructing optimal encoding trees is proposed to minimize the uncertainty and noises in the graph whilst assuring proper community partition in hierarchical abstraction. We present a novel sample-based mechanism for restoring the graph structure via node structural entropy distribution. It increases the connectivity among nodes with larger uncertainty in lower-level communities. SE-GSL is compatible with various GNN models and enhances the robustness towards noisy and heterophily structures. Extensive experiments show significant improvements in the effectiveness and robustness of structure learning and node representation learning.

Machine learning, especially deep learning, has made great achievements in application domains such as computer vision, speech recognition, machine translation, etc. This great success not only depends on the progress of model design but also on the emergence of large-scale annotated benchmark datasets. However, in some application domains, it is difficult to apply machine learning methods to solve the new arising problems due to the lack of standardized and annotated datasets. People have to find their way from the very beginning of the data collecting, cleaning, and labeling. Unfortunately, there is not a guiding framework to deal with such a situation. This paper pays attention to it and proposes a general machine learning framework to solve the pratical problems from the scratch. It contains two main stages implemented by an unsupervised clustering method and a semi-supervised learning method representatively. In the first stage, an unsupervised clustering method is utilized to find representative data samples which are more important and can be used to do data cleaning and manual labeling. In the second stage, a semi-supervised method is adopted to predict the labels for the rest data samples and to construct a larger annotated dataset. A case study in the texture perception field has been done to confirm the effectiveness and efficiency of the proposed application framework.

Given a convex and differentiable objective Q(M) for a real symmetric matrix M in the positive definite (PD) cone-used to compute Mahalanobis distances-we propose a fast general metric learning framework that is entirely projection-free. We first assume that M resides in a space S of generalized graph Laplacian matrices corresponding to balanced signed graphs. M ∈ S that is also PD is called a graph metric matrix. Unlike low-rank metric matrices common in the literature, S includes the important diagonal-only matrices as a special case. The key theorem to circumvent full eigen-decomposition and enable fast metric matrix optimization is Gershgorin disc perfect alignment (GDPA): given M ∈ S and diagonal matrix S, where Sii = 1/vi and v is the first eigenvector of M, we prove that Gershgorin disc left-ends of similarity transform B = SMS-1 are perfectly aligned at the smallest eigenvalue λmin. Using this theorem, we replace the PD cone constraint in the metric learning problem with tightest possible linear constraints per iteration, so that the alternating optimization of the diagonal / off-diagonal terms in M can be solved efficiently as linear programs via the Frank-Wolfe method. We update v using Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) with warm start as entries in M are optimized successively. Experiments show that our graph metric optimization is significantly faster than cone-projection schemes, and produces competitive binary classification performance.

Awareness to a vehicle's surrounding is necessary for safe driving. Current surround technologies focus on the detection of obstacles in hard-to-view places but may neglect temporal information. This paper seeks the causes of dangerous situations by examining surround behavior. A general hierarchical learning framework is introduced to automatically learn surround behaviors. By observing motion trajectories during natural driving, models of rear vehicle behaviors are obtained in an unsupervised fashion. The extracted behaviors are shown to correspond to typical driving scenarios, vehicle overtake and surround overtake, demonstrating the effectiveness of the learning framework.

We present a general framework for scalable cost-sensitive learning based on ensembles of classifiers. To compute ensembles for scalable learning, base models are computed from data subsets and their predictions are combined. Previous research mainly focus on combining class labels for cost-insensitive problems. In our study, we concentrate on how to combine the probability and benefit outputs (such as charity donation amount) for scalable cost-sensitive learning. We identify a few properties of both averaging and cost-sensitive optimal decision-making. Due to these properties, averaging has great potential for higher benefit as well as its obvious advantage of scalability. Experiments have shown that the averaging methods achieve a benefit level as good as or even better than the respective single classifier trained from the same dataset as a whole. However, the averaging methods exhibit linear speedup and scaled speedup that are independent of learning algorithms. Besides the averaging methods, we have experimented with tree-structured methods using variations of meta-learning and regression. Nonetheless, these approaches do not lead to higher benefits but consume higher overhead, hence are less scalable. Compared with tree-structured methods, the averaging methods are more efficient to classify examples, and easer to implement and verify.

A general learning framework which uses random sets is introduced for solving discrete-space classification problems. This framework is based on the pac-learning formalism introduced by Valiant (1984) and generalized in set-theoretic terms by Blumer, et al., (1989). The random set version of this theory is used to develop an algorithm which is a particularly efficient search scheme. This is accomplished by recasting the representational class and constructive proof presented in Valiant (1984) into random set terms and implementing it as an exhaustive search algorithm. The algorithm is a problem-specific incremental (psi) approach in that it satisfies learnability criteria for distribution-specific problems as examples are being sampled. Some theoretical and empirical analyses are presented to demonstrate the convergent pac-learnability and sample complexity of this psi-algorithm. Its performance is then tested on the multiplexor class of problems. This class has been analyzed by others as a benchmark for decision trees and genetic classifiers. Results from these test cases show that, despite using an exhaustive search, this random set implementation is computationally competitive with these more established methods (which use empirically proven heuristics). Conclusions are drawn about potential further improvements in the efficiency of this approach.

An approach to learning based on structural composition provides the basis for much of my current research work. The focus is on how existing structural elements can be composed to form larger structural elements. The importance of uniformity of representation under composition is stressed. A general learning framework is discussed which includes mechanisms for automatically proposing new compositions. Filters are used to select only the best of the proposed compositions. This framework is currently being explored in the domain of puzzle solving, where operators are the elements being composed to form macro-operators. This work should have important implications for automated mathematical discovery, automatic programming, and concept learning.

Training feedforward neural nets in Hopfield-energy-based configuration: A two-step approach

Hierarchies serve as browsing tools to access information in document collections. This article explores techniques to derive browsing hierarchies that can be used as an information map for task-based search. It proposes a novel minimum-evolution hierarchy construction framework that directly learns semantic distances from training data and from users to construct hierarchies. The aim is to produce globally optimized hierarchical structures by incorporating user-generated task specifications into the general learning framework. Both an automatic version of the framework and an interactive version are presented. A comparison with state-of-the-art systems and a user study jointly demonstrate that the proposed framework is highly effective.

Automata learning has been successfully applied in the verification of hardware and software. The size of the automaton model learned is a bottleneck for scalability, and hence optimizations that enable learning of compact representations are important. This paper exploits monads, both as a mathematical structure and a programming construct, to design and prove correct a wide class of such optimizations. Monads enable the development of a new learning algorithm and correctness proofs, building upon a general framework for automata learning based on category theory. The new algorithm is parametric on a monad, which provides a rich algebraic structure to capture non-determinism and other side-effects. We show that this allows us to uniformly capture existing algorithms, develop new ones, and add optimizations.

Outlier-resisting graph embedding

A challenge for mining large-scale streaming data overlooked by most existing studies on online learning is the skewdistribution of examples over different classes. Many previous works have considered cost-sensitive approaches in an online setting for streaming data, where fixed costs are assigned to different classes, or ad-hoc costs are adapted based on the distribution of data received so far. However, it is not necessary for them to achieve optimal performance in terms of the measures suited for imbalanced data, such as Fmeasure, area under ROC curve (AUROC), area under precision and recall curve (AUPRC). This work proposes a general framework for online learning with imbalanced streaming data, where examples are coming sequentially and models are updated accordingly on-the-fly. By simultaneously learning multiple classifiers with different cost vectors, the proposed method can be adopted for different target measures for imbalanced data, including F-measure, AUROC and AUPRC. Moreover, we present a rigorous theoretical justification of the proposed framework for the F-measure maximization. Our empirical studies demonstrate the competitive if not better performance of the proposed method compared to previous cost-sensitive and resampling based online learning algorithms and those that are designed for optimizing certain measures.

In three experiments, we studied the relation between degree of implicit learning and two aspects of short-term memory: (1) the activation level of the to-be-learned information, and (2) individual differences in short-term memory capacity. In all the experiments, we used the Nissen and Bullemer (1987) serial reaction time paradigm or a modification thereof. The effects of activation level were assessed by experimentally manipulating the rate of presentation. Individual differences in short-term memory capacity were assessed via traditional span measures. The experiments demonstrated that the rate of presentation reliably affected an indirect measure of learning (i.e., response time) under both incidental and intentional task instructions and under both single-task and dual-task conditions. Short-term memory span was reliably related to the indirect measure of learning only in some experimental conditions. The findings represent important constraints for models of implicit serial learning and are discussed within a general framework for understanding implicit learning and memory.

The purpose of this research is to explore and develop the potential of a location-based mobile application to support informal learning in the cultural heritage domain. Since the notion that learning is interweaved with people's daily lives, and also that learners get motivated to learn in different contexts and spaces while they are moving, mobile learning has emerged to support learning throughout a lifetime. This research is going to investigate how mobile technology could be developed to support informal learning with respect to cultural heritage sites, and how learners use mobile technology for learning purposes. A major contribution of this research is to develop a task model for mobile location-based learning in order to provide a general framework for informal learning in the cultural heritage domain. Furthermore, this research will use mixed methods (e.g. Interview, questionnaire and focus group) in order to gather the requirements of users and also the officials who have the responsibility for managing and maintaining cultural heritage sites. Finally, a guideline will be provided for researchers who are working in the cultural heritage domain.