An Incremental Learning Algorithm for Resource Allocating Networks Based on Local Linear Regression

Double-smoothing for bias reduction in local linear regression

A novel approach for real-time monitoring of long-term voltage stability using local linear regression and adaptive training database is proposed in this paper. Comparing to previous methods, the proposed local predictive regression model could properly balance the simplicity/transparency and accuracy, and it is also adaptive to the changes of system and operating conditions. The approach is tested on IEEE 30-bus system. Preliminary results show that it can be successfully employed in online voltage stability monitoring.

The quality of object proposal plays an important role in boosting the performance of many computer vision tasks, such as, object detection and recognition. Due to the absence of manually annotated bounding-box in practice, the quality metric towards blind assessment of object proposal is highly desirable for singling out the optimal proposals. In this paper, we propose a blind proposal quality assessment algorithm based on the Deep Objectness Representation and Local Linear Regression (DORLLR). Inspired by the hierarchy model of the human vision system, a deep convolutional neural network is developed to extract the objectness-aware image feature. Then, the local linear regression method is utilized to map the image feature to a quality score, which tries to evaluate each individual test window based on its k-nearest-neighbors. Experimental results on a large-scale IoU labeled dataset verify that the proposed method significantly outperforms the state-of-the-art blind proposal evaluation metrics.

This paper proposes a radial basis function (RBF) neural network-based model for short-term solar power prediction (SPP). Instead of predicting solar power directly, the model predicts transmissivity, which is then used to obtain solar power according to the extraterrestrial radiation. The proposed model uses a novel two-dimensional (2D) representation for hourly solar radiation and uses historical transmissivity, sky cover, relative humidity and wind speed as the input. Simulation studies are carried out to validate the proposed model for short-term SPP by using the data obtained from the National Solar Radiation Database (NSRDB). The performance of the RBF neural network is compared with that of two linear regression models, i.e., an autoregressive (AR) model and a local linear regression (LLR) model. Results show that the RBF neural network significantly outperforms the AR model and is better than the LLR model. Furthermore, the use of transmissivity and other meteorological variables, especially the sky cover, can significantly improve the SPP performance.

Optimal bandwidths for local polynomial regression usually involve functionals of the derivatives of the unknown regression function. In the multivariate case, estimates of these functionals are not readily available, primarily because estimating multivariate derivatives is complicated. In this paper, an estimator of multivariate second derivative is obtained via local quadratic regression with cross terms left out. This estimator has the optimal rate of convergence but is simpler and uses a lot less computing time than the full local quadratic estimator. Using this as a pilot estimator, an estimator of the integrated squared Laplacian of a multivariate regression function is obtained which leads to a plug-in formula of the optimal bandwidth for multivariate local linear regression. This bandwidth has good theoretical properties as well as satisfactory performance in our simulation study. It is also recommended for variable selection methods.

Increased accuracy of geostatistical prediction of nitrogen dioxide in the United Kingdom with secondary data

Asymptotic normality for a local composite quantile regression estimator of regression function with truncated data

Local linear regression models, a kind of nonparametric structures that locally perform a linear estimation of the target function, are analyzed in the context of empirical risk minimization (ERM) for function learning. The analysis is carried out with emphasis on geometric properties of the available data. In particular, the discrepancy of the observation points used both to build the local regression models and compute the empirical risk is considered. This allows to treat indifferently the case in which the samples come from a random external source and the one in which the input space can be freely explored. Both consistency of the ERM procedure and approximating capabilities of the estimator are analyzed, proving conditions to ensure convergence. Since the theoretical analysis shows that the estimation improves as the discrepancy of the observation points becomes smaller, low-discrepancy sequences, a family of sampling methods commonly employed for efficient numerical integration, are also analyzed. Simulation results involving two different examples of function learning are provided.

In this paper, we present a novel nonparametric algorithm for short term electricity demand forecasting. The algorithm is based on local linear regression using sliding window with variable length. The method for selecting optimal window length for each local fit offers close insight into trade-off between bias and standard deviation of local regressions. Optimal window length is selected for each value in the load time-series: large window for linear change of load to reduce variability and small window when load departs from linear function to control bias. In the presented algorithm local linear regression is used to estimate trend component of the load time series and to forecast trend component by extrapolating with the fitted local linear function. Some features of the algorithm are demonstrated in the paper using examples from the historic load data recorded in the Namibian Power Utility.

In this paper, we propose a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) and multiple kernel learning (MKL) based multi-modal affect recognition scheme (LSTM-MKL). It takes the LSTM-RNN advantage to model the long range dependencies between successive observations, and uses the MKL power to model the non-linear correlations between the inputs and outputs. For each of the affect dimensions (arousal, valence, expectancy, and power), two LSTM-RNN models are trained, one for each modality. In the recognition phase, the audio and visual features are input to the corresponding learned LSTM models, which in turn produce initial estimates of the affect dimensions. The LSTM outputs are further input into a multi-kernel support vector regression (MK-SVR) for the final recognition. Experimental results carried out on the AVEC2012 database, show that compared to the traditional SVR-LLR (Support Vector Machine - local linear regression) or MK-SVR fusion scheme, the proposed LSTM-MKL fusion scheme obtains higher recognition results, with an correlation coefficient (COR) of 0.354, compared to a COR of 0.124 for SVR-LLR, and 0.168 for MK-SVR, respectively.

Optical See-Through displays (OST Displays) require the appearance of the virtual content to closely resemble that of the real world. However, current OST displays typically show color distortion due to hardware limitations. To overcome this problem, we propose a chromaticity based local linear regression model for color distortion estimation of OST displays. From our initial experiments, it was observed that local samples with similar chromaticity can be modeled more linearly rather than simple nearest neighbors in a color space. We argue that the accuracy of local linear regression can be further improved by placing emphasis on similarity of chromaticity. Experimental results showed that the proposed chromaticity based color correction achieves higher accuracy compared to the nearest neighbor method, for any training set size or neighborhood size as well as global regression. It confirms that chromaticity based nearest neighbors can be modeled more linearly together than simple nearest neighbors in a color space.

We suggest a way of reducing the very high dimension of a functional predictor, X, to a low number of dimensions chosen so as to give the best predictive performance. Specifically, if X is observed on a fine grid of design points t 1 ,...,t r , we propose a method for choosing a small subset of these, say t i1 ,..., t ik , to optimize the prediction of a response variable, Y. The values t ij are referred to as the most predictive design points, or covariates, for a given value of k, and are computed using information contained in a set of independent observations (X i , Y i ) of (X, Y). The algorithm is based on local linear regression, and calculations can be accelerated using linear regression to preselect the design points. Boosting can be employed to further improve the predictive performance. We illustrate the usefulness of our ideas through simulations and examples drawn from chemometrics, and we develop theoretical arguments showing that the methodology can be applied successfully in a range of settings.

In practice, we may want to discover if there is a relationship between a response variable as a function of the predictor variables. Multiple linear regression (MLR) is a popular tool for such purpose. When the relationship is nonlinear, nonparametric regression methods such as local linear regression, smoothing splines, and support vector regression (SVR) provide flexible alternatives to MLR. How do we compare the performance of these methods and choose an appropriate one for use? In this research, we propose a general procedure to evaluate the performance of different regression methods for use in large data. We also propose an ensemble SVR for regression analysis. The proposed methods are applied to address research questions using the Research and Development Survey, conducted by the National Center for Health Statistics.

Direct regressions for underwater acoustic source localization in fluctuating oceans

Nonparametric regression is a standard statistical tool with increased importance in the Big Data era. Boundary points pose additional difficulties but local polynomial regression can be used to alleviate them. Local linear regression, for example, is easy to implement and performs quite well both at interior and boundary points. Estimating the conditional distribution function and/or the quantile function at a given regressor point is immediate via standard kernel methods but problems ensue if local linear methods are to be used. In particular, the distribution function estimator is not guaranteed to be monotone increasing, and the quantile curves can “cross.” In the article at hand, a simple method of correcting the local linear distribution estimator for monotonicity is proposed, and its good performance is demonstrated via simulations and real data examples. Supplementary materials for this article are available online.