Causal factorization machine for robust recommendation

We present an algorithmic framework for learning local causal structure around target variables of interest in the form of direct causes/effects and Markov blankets applicable to very large data sets with relatively small samples. The selected feature sets can be used for causal discovery and classification. The framework (Generalized Local Learning, or GLL) can be instantiated in numerous ways, giving rise to both existing state-of-the-art as well as novel algorithms. The resulting algorithms are sound under well-defined sufficient conditions. In a first set of experiments we evaluate several algorithms derived from this framework in terms of predictivity and feature set parsimony and compare to other local causal discovery methods and to state-of-the-art non-causal feature selection methods using real data. A second set of experimental evaluations compares the algorithms in terms of ability to induce local causal neighborhoods using simulated and resimulated data and examines the relation of predictivity with causal induction performance. Our experiments demonstrate, consistently with causal feature selection theory, that local causal feature selection methods (under broad assumptions encompassing appropriate family of distributions, types of classifiers, and loss functions) exhibit strong feature set parsimony, high predictivity and local causal interpretability. Although non-causal feature selection methods are often used in practice to shed light on causal relationships, we find that they cannot be interpreted causally even when they achieve excellent predictivity. Therefore we conclude that only local causal techniques should be used when insight into causal structure is sought. In a companion paper we examine in depth the behavior of GLL algorithms, provide extensions, and show how local techniques can be used for scalable and accurate global causal graph learning.

Bearing is an important component in mechanical equipment. Its main function is to support the rotating mechanical body and reduce the friction coefficient and axial load. In the actual operating environment, the bearings are affected by complex working conditions and other factors. Therefore, it is very difficult to effectively obtain data that meets the conditions of independent and identical distribution of training data and test data, which result in unsatisfactory fault diagnosis results. As a transfer learning method, joint distribution adaptive (JDA) can effectively solve the learning problem of inconsistent distribution of training data and test data. In this paper, a new bearing fault diagnosis method based on JDA and deep belief network (DBN) with improved sparrow search algorithm (CWTSSA), namely JACADN is proposed. In the JACADN, the JDA is employed to carry out feature transfer between the source domain samples and target domain samples, that is, the source domain samples and target domain samples are mapped into the same feature space by the kernel function. Then the maximum mean difference is used as the metric to reduce the joint distribution difference between the samples in the two domains. Aiming at the parameter selection of the DBN, an improved sparrow search algorithm (CWTSSA) with global optimization ability is used to optimize the parameters of the DBN in order to construct an optimized DBN model. The obtained source domain samples and target domain samples are divided into training set and test set, which are input the optimized DBN to construct a bearing fault diagnosis model for improving the diagnosis accuracy. The effectiveness of the proposed method is verified by vibration data of QPZZ-II rotating machinery. The experimental results show that the proposed JACADN method can effectively improve the fault diagnosis accuracy of rolling bearings under variable operating conditions.

Causal feature selection using a knowledge graph combining structured knowledge from the biomedical literature and ontologies: A use case studying depression as a risk factor for Alzheimer’s disease

SummaryAccurate prediction of the solubility of chemical substances in solvents remains a challenge. The sparsity of high-quality solubility data is recognized as the biggest hurdle in the development of robust data-driven methods for practical use. Nonetheless, the effects of the quality and quantity of data on aqueous solubility predictions have not yet been scrutinized. In this study, the roles of the size and the quality of data sets on the performances of the solubility prediction models are unraveled, and the concepts of actual and observed performances are introduced. In an effort to curtail the gap between actual and observed performances, a quality-oriented data selection method, which evaluates the quality of data and extracts the most accurate part of it through statistical validation, is designed. Applying this method on the largest publicly available solubility database and using a consensus machine learning approach, a top-performing solubility prediction model is achieved.

Traditional text classification techniques are based on a basic assumption that the underlying distributions of training and test data should be identical. However, in many real world applications, this assumption is not often satisfied. Labeled training data are expensive, but there may be some labeled data available in a different but related domain from test data. Therefore, how to make use of labeled data from a different domain to supervise the classification becomes a crucial task. In this paper, we propose a novel algorithm for cross-domain text classification using reinforcement learning. In our algorithm, the training process is iteratively reinforced by making use of the relations between documents and words. Empirically, our method is an effective and scalable approach for text categorization when the training and test data are from different but related domains. The experimental results show that our algorithm can achieve better performance than several state-of-art classifiers.

Objective: To identify optimum features from the initial feature set to classify ship ISAR images with good accuracy. The optimum feature selection brings down the computational complexity and also removes the redundant features in the feature vector. Methodology: Colour moments computed from R, G, B, RG, GB, RB component images are taken as one feature vector. The average wave energy values of horizontal, vertical and diagonal details of images decomposed up to five levels using wavelet transform are considered as second feature vector. The ship ISAR images measured with high resolution Radar are used as data set to test the classification performance of the two feature sets, which is explored uniquely in this work. Genetic algorithm and particle swarm optimization techniques are used to find the optimum feature sets. Probabilistic neural network is used to compare the classification accuracy achieved with full length feature vectors and optimum feature vectors of the two considered feature sets. Findings: Data set of two ships is taken. The data of each ship is divided in to two parts training data and test data. The feature vectors computed from training set are used to train the classifier and that computed from test set are used to test the classification accuracy. The number of variables (number of features in the feature vector) of optimization algorithms Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) is varied to find the optimum feature set to classify the ship ISAR image data. The function that computes the classification accuracy is used as the objective function of the optimization algorithms. From the initial feature vector of thirty colour moments a set of four features is identified that gives 100% accuracy for both the targets considered. From starting set of fifteen average energy levels, a set of four energy levels identified by PSO and the set of three energy levels are found to be more effective in classification of the ship ISAR images with classification accuracy of 66.6% for the first target and 75% for the second target. Both the optimization techniques performed in similar way. Other optimization algorithms can be explored to find feature combinations that classify ship ISAR images with better accuracy. Application: Classification based on ISAR images has application in automatic non-cooperative target recognition, development of monitoring systems for surveillance along transportation corridors etc. to avoid illegal immigration. Future Work: Different classifiers Self Organizing Maps, Support Vector Machines etc. can be used to study the possibility of obtaining optimum solution in smaller time. Keywords: Classification, Feature Selection, ISAR image, Optimum Feature Set, Optimization Techniques

At present, there are two main problems in the commonly used radar emitter identification methods. First, when the distribution of training data and testing data is quite different, the identification accuracy is low. Second, the traditional identification methods usually include an offline training stage and online identifying stage, which cannot achieve the real-time identification of the radar emitter. Aimed at the above problems, this paper proposes a radar emitter identification method based on transfer learning and online learning. First, for the case where the target domain contains only a small number of labeled samples, the TrAdaBoost method is used as the basic learning framework to train a support vector machine, which can obtain useful knowledge from the source domain to aid in the identification of the target domain. Then, for the case where the target domain does not contain labeled samples, the Expectation-Maximization algorithm is used to filter the unlabeled samples in the target domain to generate the available training data. Finally, to make the identification quickly and accurately, we propose a radar emitter identification method, based on online learning to ensure real-time updating of the model. Simulation experiments show that the proposed method, based on transfer learning and online learning, has higher identification accuracy and good timeliness.

This focus issue of Physiological Measurement follows the 38th Annual International Computing in Cardiology (CinC) Conference, hosted in Hangzhou, China in September 2011 by Zhejiang University. Each year, the NIH-sponsored PhysioNet resource (http://physionet.org/) runs an open competition lasting several months, aimed at encouraging the development of solutions to an unsolved or poorly solved problem in biomedicine, in most cases making use of relevant clinical and experimental data provided freely by PhysioNet. Participants in these annual challenges discuss their diverse approaches to the Challenge problems during dedicated scientific sessions at CinC. The topics of these PhysioNet/CinC Challenges range from physiologic signal processing and analysis to forecasting and modelling clinically important events and processes.

Joint nonlinear feature selection and continuous values regression network

Domain adaptation techniques such as importance weighting modify the training data to better represent a different test data distribution, a process that may be particularly vulnerable to a malicious attack in an adversarial machine learning scenario. In this work, we explore the level of such vulnerability of importance weighting to poisoning attacks. Importance weighting, like other domain adaptation approaches, assumes that the distributions of training and test data are different but related. An intelligent adversary, having full or partial access to the training data, can take advantage of the expected difference between the distributions, and can inject well crafted malicious samples into the training data, resulting in an incorrect estimation of the importance ratio. In this work, we demonstrate the vulnerability of one of the simplest yet most effective approaches for directly estimating the importance ratio, namely, modifying the training distribution using a discriminative classifier such as the logistic regression. We test the robustness of the importance weighting process using well-controlled synthetic datasets, with an increasing number of attack points in the training data. Under the worst case perfect knowledge scenario, where the attacker has full access to the training data, we demonstrate that importance weighting can be dramatically compromised with the insertion of even a single attack point. We then show that even under limited knowledge scenario, where the attacker has limited access to the training data, the estimation process can still be significantly compromised.

Extreme weather modeling is a challenge for modeling experts in Indonesia and the world. Extreme weather prediction is a complex problem because the chances of it happening are very small, so the developed models often have a low level of accuracy. The purpose of this research is to combine the classic model, Autoregressive Integrated Moving Average (ARIMA), recurrent neural network (RNN) model using Adam and SGD estimation (RNN-Adam and RNN-SGD) with the reLU, tanh, sigmoid and gaussian activation functions. In addition, the ARIMA-RNN mix model was also demonstrated in this study. These models are applied to monthly period extreme weather data obtained from the Meteorology, Climatology and Geophysics Agency (BMKG) of West Sulawesi Province which are converted into training data and test data. The RMSE value is used to see the level of prediction accuracy in both training data and test data. Based on the research results, the best model obtained for modeling Indonesia’s extreme weather is the ARIMA-RNN-Adam mix model with the reLU activation function based on the RMSE value on the training and test data. At n = 50, the smallest RMSE and MSE values of the third model are the ARIMA-RNN-Adam model which is 0.23212 using the reLU activation function, then the ARIMA-RNN-SGD model which is 0.25432 with the same activation function, while the ARIMA value is 0.3270. At n=100 it can be seen that the smallest RMSE and MSE values of the three models are the ARIMA-RNN-Adam model which is equal to 0.25149 using the reLU activation function, then the ARIMA-RNN-SGD model which is equal to 0.25256 with the same activation function, while the ARIMA value is 0.2644.

Face Anti-Spoofing (FAS) is essential to secure face recognition systems and has been extensively studied in recent years. Although deep neural networks (DNNs) for the FAS task have achieved promising results in intra-dataset experiments with similar distributions of training and testing data, the DNNs' generalization ability is limited under the cross-domain scenarios with different distributions of training and testing data. To improve the generalization ability, recent hybrid methods have been explored to extract task-aware handcrafted features (e.g., Local Binary Pattern) as discriminative information for the input of DNNs. However, the handcrafted feature extraction relies on experts' domain knowledge, and how to choose appropriate handcrafted features is underexplored. To this end, we propose a learnable network to extract Meta Pattern (MP) in our learning-to-learn framework. By replacing handcrafted features with the MP, the discriminative information from MP is capable of learning a more generalized model. Moreover, we devise a two-stream network to hierarchically fuse the input RGB image and the extracted MP by using our proposed Hierarchical Fusion Module (HFM). We conduct comprehensive experiments and show that our MP outperforms the compared handcrafted features. Also, our proposed method with HFM and the MP can achieve state-of-the-art performance on two different domain generalization evaluation benchmarks.

As a supervised learning method, Factorization Machine (FM) is famous for its capability of modeling feature interactions. However, FM’s performance might be bad if we assign the same weight to all feature interactions, as not all of them are equally useful and productive. Attentional Factorization Machine (AFM) improves FM by discriminating the importance of distinctive feature interactions via a neural attention network. Nevertheless, the neural attention network in AFM is not fine-grained enough and it ignores the information of the fields implied by the features, which limits the performance of the model. In this work, we propose a novel model named IO-aware Factorization Machine (IOFM), which enhances the feature representation ability of attention mechanism in estimating weights via two awareness auxiliary matrices. To make the model more efficient, we further reduce the model parameters using canonical decomposition for the two auxiliary matrices and design a shared matrix to correlate the decomposed matrices. Extensive experiments on two real-world datasets indicate the superiority of our IOFM model over the state-of-the-art methods.

Causal Feature Selection in Neuroscience

Financial institutions have a large amount of data on their borrowers, which can be used to predict the probability of borrowers defaulting their loan or not. Some of the models that have been used to predict individual loan defaults include linear discriminant analysis models and extreme value theory models. These models are parametric in nature since they assume that the response being investigated takes a particular functional form. However, there is a possibility that the functional form used to estimate the response is very different from the actual functional form of the response. The purpose of this research was to analyze individual loan defaults in Kenya using the logistic regression model. The data used in this study was obtained from equity bank of Kenya for the period between 2006 to 2016. A random sample of 1000 loan applicants whose loans had been approved by equity bank of Kenya during this period was obtained. Data obtained was on the credit history, purpose of the loan, loan amount, nature of the saving account, employment status, sex of the applicant, age of the applicant, security used when acquiring the loan and the area of residence of the applicant (rural or urban). This study employed a quantitative research design, it deals with individual loans defaults as group characteristics of a borrower. The data was pre-processed by seeding using R- Software and then split into training dataset and test data set. The train data was used to train the logistic regression model by employing Supervised machine learning approach. The R-statistical software was used for the analysis of the data. The test data set was used to do cross-validation of the developed logistic model which later was used for analysis prediction of individual loan defaults. This study focused on the analysis of individual loan defaults in Kenya using the logistic regression model in Machine learning. The logistic regression model predicted 303 defaults from train data set, 122 non-defaults and misclassified loans were 56 and 69. The model had an accuracy of 0.7727 with the train data and 0.7333 with the test data. The logistic regression model showed a precision of 0.8440 and 0.8244 with the train and test data respectively. The performance of the model with both the train and test data was illustrated using a plot of train errors and test errors against sample size on the same axes. The plot showed that the performance of the model increases with an increase in sample size. The study recommended the use of logistic regression in conjunction with supervised machine learning approach in loan default prediction in financial institutions and also more research should be carried out on ensemble methods of loan defaults prediction in order to increase the prediction accuracy.