Orthogonal Least Square RBF Based Implicit Surface Reconstruction Methods

<p>An investigation into optimized support vector machine (SVM) integrated with principal component analysis (PCA) and orthogonal least square (OLS) in classifying asphyxiated infant cry was performed in this study. Three approaches were used in the classification; SVM, PCA-SVM, and OLS-SVM. Various numbers of features extracted from Mel-frequency Cepstral coefficient (MFCC) were tested to obtain the optimal parameters of SVM kernels. Once the optimal feature set is obtained, PCA and OLS selected the most significant features and the optimized SVM then classified the selected cry patterns. In PCA-SVM, eigenvalue-one-criterion (EOC), cumulative percentage variance (CPV) and the Scree test (SCREE) were used to select the most significant features. SVM with radial basis function (RBF) kernel was chosen in the classification stage. The classification accuracy and computation time were computed to evaluate the performance of each method. The best method for classifying asphyxiated infant cry is PCA-SVM with EOC since it produces the highest classification accuracy which is 94.84%. Using PCA-SVM, the classification process was performed in 1.98s only. The results also show that employing feature selection techniques could enhance the classifier performance.</p>

Orthogonal least squares (OLS) is a classic algorithm for sparse recovery, function approximation, and subset selection. In this paper, we analyze the performance guarantee of the OLS algorithm. Specifically, we show that OLS guarantees the exact reconstruction of any K-sparse vector in K iterations, provided that a sensing matrix has unit ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm columns and satisfies the restricted isometry property (RIP) of order K + 1 with δ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K+1</sub> <; C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</sub> = {1/√K, K = 1, 1/√(K + 1/4), K = 2, 1/√(K + 1/16), K = 3, 1/√K, K ≥ 4}. Furthermore, we show that the proposed guarantee is optimal in the sense that if δ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K+1</sub> ≥ C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</sub> , then there exists a counterexample for which OLS fails the recovery.

Orthogonal least squares based fast feature selection for linear classification

This paper describes an application of the Orthogonal Least Squares (OLS) algorithm for feature selection of spoken letters. Traditionally used for system identification purposes, the OLS method was used to select important Mel-Frequency Cepstrum Coefficients (MFCC) for classification of two spoken letters - `A' and `S' using Multi-Layer Perceptron (MLP) neural network. We evaluated several network structures and parameters to determine the best performance in terms of accuracy and speed. The result found that OLS is an effective feature selection method, with the best classification performance of 85% with 6 hidden units.

In this work, we propose a paradigm for constructing a sparsely-connected multi-layer perceptron (MLP). Using Orthogonal Least Squares (OLS) method for training, the proposed method prunes the hidden units and output weights based on their usefulness to design a sparsely connected MLP. We formulate second order algorithm to obtain the closed-form expression for hidden unit learning factors thereby minimizing hand-tuned parameters. The usefulness of the proposed algorithm is further substantiated by its ability to differentiate two combined datasets. Using widely available datasets, the proposed algorithm's 10-fold testing error is shown to be less than that of several other algorithms. Inducing sparsity into a fully-connected neural network, pruning of the hidden units, Newton's method for optimization, and orthogonal least squares are the subject matter of the present work.

The Orthogonal Least Squares (OLS) algorithm is an efficient sparse recovery algorithm that has received much attention in recent years. On one hand, this paper considers that the OLS algorithm recovers the supports of sparse signals in the noisy case. We show that the OLS algorithm exactly recovers the support of $K$-sparse signal $\boldsymbol{x}$ from $\boldsymbol{y}=\boldsymbol{\unicode[STIX]{x1D6F7}}\boldsymbol{x}+\boldsymbol{e}$ in $K$ iterations, provided that the sensing matrix $\boldsymbol{\unicode[STIX]{x1D6F7}}$ satisfies the restricted isometry property (RIP) with restricted isometry constant (RIC) $\unicode[STIX]{x1D6FF}_{K+1}&lt;1/\sqrt{K+1}$, and the minimum magnitude of the nonzero elements of $\boldsymbol{x}$ satisfies some constraint. On the other hand, this paper demonstrates that the OLS algorithm exactly recovers the support of the best $K$-term approximation of an almost sparse signal $\boldsymbol{x}$ in the general perturbations case, which means both $\boldsymbol{y}$ and $\boldsymbol{\unicode[STIX]{x1D6F7}}$ are perturbed. We show that the support of the best $K$-term approximation of $\boldsymbol{x}$ can be recovered under reasonable conditions based on the restricted isometry property (RIP).

The prediction accuracy and generalization ability of neural/neurofuzzy models for chaotic time series prediction highly depends on employed network model as well as learning algorithm. In this study, several neural and neurofuzzy models with different learning algorithms are examined for prediction of several benchmark chaotic systems and time series. The prediction performance of locally linear neurofuzzy models with recently developed Locally Linear Model Tree (LoLiMoT) learning algorithm is compared with that of Radial Basis Function (RBF) neural network with Orthogonal Least Squares (OLS) learning algorithm, MultiLayer Perceptron neural network with error back-propagation learning algorithm, and Adaptive Network based Fuzzy Inference System. Particularly, cross validation techniques based on the evaluation of error indices on multiple validation sets is utilized to optimize the number of neurons and to prevent over fitting in the incremental learning algorithms. To make a fair comparison between neural and neurofuzzy models, they are compared at their best structure based on their prediction accuracy, generalization, and computational complexity. The experiments are basically designed to analyze the generalization capability and accuracy of the learning techniques when dealing with limited number of training samples from deterministic chaotic time series, but the effect of noise on the performance of the techniques is also considered. Various chaotic systems and time series including Lorenz system, Mackey-Glass chaotic equation, Henon map, AE geomagnetic activity index, and sunspot numbers are examined as case studies. The obtained results indicate the superior performance of incremental learning algorithms and their respective networks, such as, OLS for RBF network and LoLiMoT for locally linear neurofuzzy model.

formula omitted]-step analysis of orthogonal greedy algorithms for non-negative sparse representations

Background: Implicit surface is a kind of surface modeling tool, which is widely used in point cloud reconstruction, deformation and fusion due to its advantages of good smoothness and Boolean operation. The most typical method is the surface reconstruction with Radial Basis Functions (RBF) under normal constraints. RBF has become one of the main methods of point cloud fitting because it has a strong mathematical foundation, an advantage of computation simplicity, and the ability of processing nonuniform points. Objective: Techniques and patents of implicit surface reconstruction interpolation with RBF are surveyed. Theory, algorithm, and application are discussed to provide a comprehensive summary for implicit surface reconstruction in RBF and Hermite Radial Basis Functions (HRBF) interpolation. Methods: RBF implicit surface reconstruction interpolation can be divided into RBF interpolation under the constraints of points and HRBF interpolation under the constraints of points and corresponding normals. Results: A total of 125 articles were reviewed, in which more than 30% were related to RBF in the last decade. The continuity properties and application fields of the popular global supported radial basis functions and compactly supported radial basis functions are analyzed. Different methods of RBF and HRBF implicit surface reconstruction are evaluated, and the challenges of these methods are discussed. Conclusion: In future work, implicit surface reconstruction via RBF and HRBF should be further studied in fitting accuracy, computation speed, and other fundamental problems. In addition, it is a more challenging but valuable research direction to construct a new RBF with both compact support and improved fitting accuracy.

Due to the difficulty of establishing the accurate control model for building an intelligent system, a neural network predictive control method is proposed, in this paper, based on a weed optimization algorithm. Through considering indoor temperature and relative humidity environment factors, a control model of temperature and humidity environment is first established in an intelligent building. Then, the hidden layer nodes center of the RBF neural network is optimized by using the weed optimization algorithm. The above mentioned work focuses on improving the shortcomings of Orthogonal Least Squares (OLS) algorithm, and simultaneously simplifies the network architecture. The simulation results show that the RBF neural network predictive control method based on the weed optimization algorithm has better approximation ability and generalization ability contrasting with the OLS algorithm.

The Orthogonal Least Squares (OLS) algorithm sequentially selects columns of the coefficient matrix to greedily find an approximate sparse solution to an underdetermined system of linear equations. In this paper, conditions under which OLS recovers sparse signals from a low number of random linear measurements with probability arbitrarily close to one are stated. Moreover, a computationally efficient generalization of Orthogonal Least-Squares which relies on a recursive relation between the components of the optimal solution to select L columns at each step and solve the resulting overdetermined system of equations is proposed. This generalized OLS algorithm is empirically shown to outperform existing greedy algorithms broadly used in literature.

This paper proposes a Real-Valued Time-Delay Recurrent Radial Basis Function (RVTDRRBF) model suitable for dynamic modeling of the strongly nonlinear behaviors of the Doherty Power Amplifiers (DPAs). This model has four Tapped Delay Lines (TDLs), which account for the memory effect of the DPA. The structure of the RVTDRRBF model is simpler than the traditional FeedForward Neural Networks (FFNNs) model. Weights and centers of the proposed model can be resolved by the Orthogonal Least Square (OLS) and Singular Value De-composition (SVD) algorithm. A three-carrier Wideband Code Division Multiple Access (WCDMA) signal is taken as the test signal. The simulation results in frequency-domain and time-domain for a DPA with 51 dBm output illustrate a good agreement between the RVTDRRBF model and measurement data. Moreover, comparing the Normalized Mean Square Error (NMSE) of RVTDRRBF model, memory polynomial model and RVTDRBF model, it can be noticed that the proposed RVTDRRBF model is more accurate than the RVTDRBF model and the memory polynomial model in modeling the strong dynamic nonlinearity of the DPAs.

A new learning method of radial basis function (RBF) network based on fuzzy C-means (FCM) clustering and ant colony optimization (ACO) is presented in the paper. Generally, the crucial problem of learning RBF network is the selection of the unit number, the centers and widths of the Gaussian radial basis function in hidden layer, and the weights between hidden layer and output layer. In this work, the centers are determined by FCM clustering, and the weights are determined by least mean squares (LMS) method. In addition, ACO is introduced to optimize two important parameters of FCM including the weighting exponent which impacts its performance, and the clustering number that is equal to the hidden unit number which impacts the generalization of RBF network, by minimizing an objective function integrated the generalization error with the hidden unit number of RBF network. Simulation results of identifying a nonlinear system illustrate the effectiveness of designing the RBF network with smaller structure but stronger generalization ability, comparing with K-means and orthogonal least squares (OLS) based learning methods.

This paper introduces an interval type-2 modified fuzzy c-regression model (IT2MFCRM) clustering algorithm for identifying the structure in TS Fuzzy Model (TSFM). A scaling factor has been used for identifying the model parameters of interval type-2 fuzzy set. Once, the type-1 MFCRM clustering algorithm has been performed to obtain the premise parameters of membership function, then scaling factor has been used to obtain type-2 premise parameters. Once the type-2 membership function is obtained, then type reduction technique has been used for identifying the coefficients of consequence parameters. Orthogonal Least Square (OLS) method has been applied for determining the consequence parameters. Finally, the IT2MFCRM based TS fuzzy model has been validated on two benchmark examples.

Bioethanol production by means of anaerobic thermophilic microorganisms with pentose or hexose as the substrate are of paramount importance in sustainable fuel innovation. Manipulation of microorganisms and the associated experiment conditions by means of various ad-hoc technology is obviously the most straightforward way with the aim of maximizing bioethanol yield. However, methodology by means of mathematical modeling and analysis is often neglected among these routines. In this paper, typical input-output models are applied in the metabolic system analysis of Thermoanaerobacter sp. X514 under sole glucose substrate, sole xylose substrate and mixed glucose and xylose substrates conditions. Orthogonal Least Squares (OLS) approach is used for model parameter estimation. Model selection is proposed in order to testify the generality of the suggested model. System identification results illustrate that various forms of Nonlinear AutoRegressive with eXogenous input models (NARX) are applicable in delineating the system where different substrates (glucose or xylose) are utilized during the experiments. The proposed model structure infers that the yields of various products in X514 are mainly driven by the history information of the substrate consumption change. Moreover, the interaction between the main fermentation products of X514 is indirectly connected through the proposed models.