Optimal Margin Distribution Machine with Sparsity Inducing Penalty

Support Vector Machine (SVM) has always been one of the most successful learning algorithms, with the central idea of maximizing the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">minimum margin</i> , i.e., the smallest distance from the instances to the classification boundary. However, recent theoretical results disclosed that maximizing the minimum margin does not necessarily lead to better generalization performance, and instead, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">margin distribution</i> has been proven to be more crucial. Based on this idea, we propose the Optimal margin Distribution Machine (ODM), which can achieve a better generalization performance by optimizing the margin distribution explicitly. We characterize the margin distribution by the first- and second-order statistics, i.e., the margin mean and variance. The proposed method is a general learning approach which can be applied in any place where SVMs are used, and its superiority is verified both theoretically and empirically in this paper.

A kernel-free Laplacian quadratic surface optimal margin distribution machine with application to credit risk assessment

ε-Support vector regression (ε-SVR) is a powerful machine learning approach that focuses on minimizing the margin, which represents the tolerance range between predicted and actual values. However, recent theoretical studies have highlighted that simply minimizing structural risk does not necessarily result in well margin distribution. Instead, it has been shown that the distribution of margins plays a more crucial role in achieving better generalization performance. Furthermore, the kernel-free technique offers a significant advantage as it effectively reduces the overall running time and simplifies the parameter selection process compared to the kernel trick. Based on existing kernel-free regression methods, we present two efficient and robust approaches named quadratic hyper-surface kernel-free large margin distribution machine-based regression (QLDMR) and quadratic hyper-surface kernel-free least squares large margin distribution machine-based regression (QLSLDMR). The QLDMR optimizes the margin distribution by considering both ε-insensitive loss and quadratic loss function similar to the large-margin distribution machine-based regression (LDMR). QLSLDMR aims to reduce the cost of the computing process of QLDMR, which transforms inequality constraints into an equality constraint inspired by least squares support vector machines (LSSVR). Both models combined the spirit of optimal margin distribution with kernel-free technique and after simplification are convex so that they can be solved by some classical methods. Experimental results demonstrate the superiority of the optimal margin distribution combined with the kernel-free technique in robustness, generalization, and efficiency.

Positive-unlabeled (PU) learning deals with the circumstances where only a portion of positive instances are labeled, while the rest and all negative instances are unlabeled, and due to this confusion, the class prior can not be directly available. Existing PU learning methods usually estimate the class prior by training a nontraditional probabilistic classifier, which is prone to give an overestimation. Moreover, these methods learn the decision boundary by optimizing the minimum margin, which is not suitable in PU learning due to its sensitivity to label noise. In this paper, we enhance PU learning methods from the above two aspects. More specifically, we first explicitly learn a transformation from unlabeled data to positive data by entropy regularized optimal transport to achieve a much more precise estimation for class prior. Then we switch to optimizing the margin distribution, rather than the minimum margin, to obtain a label noise insensitive classifier. Extensive empirical studies on both synthetic and real-world data sets demonstrate the superiority of our proposed method.

A fault detection method of wind turbine pitch system using semi-supervised optimal margin distribution machine (ssODM) optimized by dynamic state transition algorithm (DSTA) [ssODM-DSTA] was proposed to solve the problem of obtaining the optimal hyperparameters of the fault detection model for the pitch system. This method was adopted to input the three hyperparameters of the ssODM into the dynamic state transition algorithm in the form of a three-dimensional vector to obtain the global optimal hyperparameters of the model, thus improving the performance of the fault detection model. Using a random forest to rank the priority of features of the pitch system fault data, the features with large weight proportions were retained. Then, the Pearson correlation method is used to analyze the degree of correlation among features, filter redundant features, and reduce the scale of features. The dataset was divided into a training dataset and a test dataset to train and test the proposed fault detection model, respectively. The real-time wind turbine pitch system fault data were collected from domestic wind farms to carry out fault detection experiments. The results have shown that the proposed method had a positive fault rate (FPR) and fault negative rate (FNR), compared with other optimization algorithms.

A biologically inspired controller for sound and vibration applications

Feature selection of generalized extreme learning machine for regression problems

We present a novel column generation based boosting method for multi-class classification. Our multi-class boosting is formulated in a single optimization problem as in [1]. Different from most existing multi-class boosting methods, which use the same set of weak learners for all the classes, we train class specified weak learners (i.e., each class has a different set of weak learners). We show that using separate weak learner sets for each class leads to fast convergence, without introducing additional computational overhead in the training procedure. To further make the training more efficient and scalable, we also propose a fast coordinate descent method for solving the optimization problem at each boosting iteration. The proposed coordinate descent method is conceptually simple and easy to implement in that it is a closed-form solution for each coordinate update. Experimental results on a variety of datasets show that, compared to a range of existing multi-class boosting methods, the proposed method has much faster convergence rate and better generalization performance in most cases. We also empirically show that the proposed fast coordinate descent algorithm needs less training time than the MultiBoost algorithm in [1].KeywordsTraining TimeColumn GenerationMaster ProblemWeak LearnerCoordinate DescentThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The research of localization technology based on received signal strength and machine learning has recently attracted a lot of attentions, since with the help of enough labeled training data this technology is able to achieve high positioning accuracy. However, it is an expensive job to collect enough labeled training data in the broad outdoor space. In order to reduce the cost of building and maintaining training database, semi-supervised extreme learning machine is applied to solve the cellular network localization in this article. However, the performance of this algorithm is sensitive to the values of the hyper parameters. Without any systematic guidance, the optimal hyper parameters can only be selected by experienced workers through trial and error. To address this problem, we propose a novel algorithm by combining particle swarm optimization and semi-supervised extreme learning machine to automatically select the optimal hyper parameters of semi-supervised extreme learning machine in this article. ...

An optimal weight learning machine for handwritten digit image recognition

Over the past decade, Smart cities have been advanced and minimizing traffic congestion become the main concern in the progression of smart cities. The rapid growth in the number of road vehicles has raised the number of road accidents and traffic congestion. To Solve this problem, Vehicular Network (VN) formulated several novel concepts which include traffic control, vehicular communications, and navigation. Machine Learning (ML) will be an effective technique for identifying hidden insights into ITS without being programmed clearly by learning from data. This article develops an Optimal Extreme Learning Machine based Traffic Congestion Control System (OELM-TCCS) in vehicular networks. The presented OELM-TCCS technique mainly focuses on the identification and recognition of traffic congestion in VANET. To perform this, the presented OELM-TCCS technique primarily designs a new ELM model to carry out the classification process. In addition, the presented OELM-TCCS technique executes the satin bowerbird optimization (SBO) algorithm for parameter tuning of the ELM method. To demonstrate the boosted performance of the OELM-TCCS model, a series of experiments were executed. The experimental outcomes indicate the betterment of the OELM-TCCS method over recent methods with maximum accuracy of 99.17%.

In this study, the authors aimed to study an effective intelligent method for employment stability prediction in order to provide a reasonable reference for postgraduate employment decision and for policy formulation in related departments. First, this paper introduces an enhanced slime mould algorithm (MSMA) with a multi-population strategy. Moreover, this paper proposes a prediction model based on the modified algorithm and the support vector machine (SVM) algorithm called MSMA-SVM. Among them, the multi-population strategy balances the exploitation and exploration ability of the algorithm and improves the solution accuracy of the algorithm. Additionally, the proposed model enhances the ability to optimize the support vector machine for parameter tuning and for identifying compact feature subsets to obtain more appropriate parameters and feature subsets. Then, the proposed modified slime mould algorithm is compared against various other famous algorithms in experiments on the 30 IEEE CEC2017 benchmark functions. The experimental results indicate that the established modified slime mould algorithm has an observably better performance compared to the algorithms on most functions. Meanwhile, a comparison between the optimal support vector machine model and other several machine learning methods on their ability to predict employment stability was conducted, and the results showed that the suggested the optimal support vector machine model has better classification ability and more stable performance. Therefore, it is possible to infer that the optimal support vector machine model is likely to be an effective tool that can be used to predict employment stability.

In this paper, an interval type-2 Takagi-Sugeno fuzzy classification system (IT2T-SFCS) learned by particle swarm optimization (PSO) and support vector machine (SVM) for antecedent and consequent parameters optimization is proposed. The IT2T-SFCS is constructed by fuzzy if-then rules whose antecedents are interval type-2 fuzzy sets and consequents are linear state equations. The antecedents of IT2T-SFCS use the fuzzy iterative self-organizing data analysis technique (ISODATA) and PSO to learn and calculate the optimal centers and the uncertain widths of the Gaussian membership functions. Consequent parameters in IT2T-SFCS are learned through SVM for the purpose of achieving higher generalization ability. The proposed IT2T-SFCS is able to directly handle uncertainties, minimize the effects of uncertainties and get the better generalization performance, which inherits the benefits of interval type-2 T-S fuzzy system and SVM. For demonstration, IT2T-SFCS is used as a classifier in gender recognition. The experimental results show that the performance of the proposed IT2T-SFCS is superior to that of the previous mainstream classifiers.

The stock price varies depending on time, so stock market data is time-series data. The prediction of the trend of a stock price is a more interesting topic for investors to take an investment decision in a specific stock. Prediction of stock price always depends on machine learning algorithms. In this work, optimizing deep neural network (DNN) is used for predicting if the close price is reached to the profit which is determined by the investor or not and improve the prediction accuracy. Particle swarm optimization (PSO) and auto machine learning (AutoML) are used as optimizers with DNNs. The methods are applied to data of nine companies in Indonesia and National Stock Exchange (NSE) of India. The data is got from yahoo finance. Based on the experimental results, AutoML of deep learning proved to have the best accuracy rate, which is varying from 81 percent to 92 percent across all companies, and the accuracy after optimizing DNNs using PSO is varying from 73 percent to 82 percent across all companies.

Aim: Given a finite set of cardinal numbers, we approximate the data by an optimal normal distribution based on some criteria. Methods: There are mainly two parts: interpolating the given data via natural spline functions and searching the optimal normal distribution that would minimize the absolute area of the difference function derived from the interpolated density function and the Gaussian distributions. The whole procedures are divided into 10 steps. Results: Our experimental results show the recovered population mean and standard derivation highly depend on the chosen samples. Nonetheless, the interpolated density functions are in the similar shapes of the normal distribution. In addition, the increase of the standard derivation doesn’t affect the approximation, though the optimal standard derivation is largely proportional to it. Conclusions: A method with a set of procedures is devised to search the optimal normal distribution for a given set of data. The proposed method takes the absolute area of the difference function into consideration. The interpolated density function is used to match the optimal normal distribution. This matching is much intuitive and straightforward. Other comparative results also show this approach is stable and intuitively justifiable.