Comparative Analysis of Neural Network and Linear Regression Applied to Black Friday Data

Space-Division Multiple Access-OFDM based wireless communication has the potential to significantly increase the spectral efficiency, system performance and number of users. Research in the development of efficient multiuser signal detection algorithms for such systems has generated much interest in recent years. This research proposes MUD schemes utilizing three neural network (NN) models like Feed Forward NN (FNN) without any hidden layer, FNN with a single hidden layer and Recurrent Neural Network (RNN) as possible alternatives to existing Genetic Algorithm (GA) based Minimum Bit Error Rate (MBER) MUD. Further, these techniques offer low complexity. Extensive simulation based performance study is carried out to prove the efficiency of the proposed techniques. Better performance of FNN with hidden layer and RNN compared to FNN without hidden layer is clearly observed in rank deficient MIMO scenario, where number of users exceeds the number of receiving antennas. The Bit Error Rate (BER) and complexity analysis of the proposed neural MUD schemes are close to optimal ML and show improvement over the previous implemented technique.

Function approximation is one of the core tasks that are solved using neural networks in the context of many engineering problems. However, good approximation results need good sampling of the data space, which usually requires exponentially increasing volume of data as the dimensionality of the data increases. At the same time, often the high-dimensional data is arranged around a much lower dimensional manifold. Here we propose the breaking of the function approximation task for high-dimensional data into two steps: (1) the mapping of the high-dimensional data onto a lower dimensional space corresponding to the manifold on which the data resides and (2) the approximation of the function using the mapped lower dimensional data. We use over-complete self-organizing maps (SOMs) for the mapping through unsupervised learning, and single hidden layer neural networks for the function approximation through supervised learning. We also extend the two-step procedure by considering support vector machines and Bayesian SOMs for the determination of the best parameters for the nonlinear neurons in the hidden layer of the neural networks used for the function approximation. We compare the approximation performance of the proposed neural networks using a set of functions and show that indeed the neural networks using combined unsupervised and supervised learning outperform in most cases the neural networks that learn the function approximation using the original high-dimensional data.

<div class="section abstract"><div class="htmlview paragraph">Virtual sensing or estimation of emission species such as NOx at the engine exhaust using the appropriate engine measurables and leveraging it for control/diagnosis is a challenging task given the highly nonlinear and dynamic nature of the combustion process. This article presents development of virtual engine-out NOx (EONOx) sensor using two different supervised dynamic artificial neural network (ANN) topologies, namely, trained recurrent neural network (RNN) and wavelet neural network (WNN). The proposed RNN architecture is a single hidden layer neural network with permutations of feedback connections between the inter- and intra-layer nodes. The RNN resembles a nonlinear state-space model mapping select engine measurables and the engine-out NOx and is trained using a variant of real-time recurrent learning (RTRL) algorithm. The WNN architecture is a single hidden layer neural network comprising hidden layer nodes with wavelets as activation functions. The activation functions of the WNN nodes are adapted for their form and time shift along with their synaptic weights in the supervised learning method. The topologies are validated in virtual environment using modeled data as well as experimental data. Approaches toward leveraging these virtual sensors for better NOx control, both at the engine-out and system-out level are discussed along with their benefits. The limitations of such data-based virtual sensors are stated. The outcome of this work is methodology to select appropriate ANN topology and training it for efficient EONOx virtual sensor and leveraging it for control at engine and tailpipe level.</div></div>

A multilayer feedforward neural network with a modification to the standard backpropagation training of neural nets is considered. Information on the gradients of target outputs with respect to network inputs is used in this modification. This modified neural network is then applied for approximating design requirements of aerospace composite components, such as an aircraft engine guide vane and a satellite reflector assembly. This neural network functional approximator is then used with an optimization algorithm for determining the optimal parameters for component design requirements. The modified neural network provides for faster convergence of learning process and requires a smaller number of hidden layer nodes for convergence to similar training error levels. HE application of neural networks as universal approximators has been the focus of tremendous amount of activity in the past few years.16 Neural networks have been used for approximating and modeling both memoryless systems and dynamical systems with memory. This paper does not go into the various neural net architectures and training schemes, and the interested reader could refer to standard texts on this subject matter.1'2 This paper considers static neural networks (systems without memory) formed by interconnections without loops, called feedforward networks.2 In addition, this work considers a backpropagation learning scheme with adaptation of learning rate. There have been many additions to, or adaptations of, the basic backpropagation scheme, such as adaptive learning rate, learning with momentum, and the like,1'2 for improving convergence of the learning scheme, and these are not addressed in this paper. This paper explores a modification of the backpropagation learning scheme using information on the gradient of the target outputs with respect to the inputs of the network. The performance of the modified learning scheme is compared with that of the standard backpropagation learning scheme.2 A feedforward network with a single hidden layer trained with this modified learning scheme is applied in structural optimization problems. Figure 1 shows a feedforward net with a single hidden layer. The restriction to a single hidden layer is not a limitation of the modified learning scheme, and the new training scheme can be used on networks with multiple hidden layers also. The use of neural networks for structural response approximation has been reported by several authors.46 The primary reason for investigating such properties is to assess the applicability of trained neural networks for quick and accurate approximation of response functions during structural design optimization. Structural optimization, utilizing either gradient-based numerical search or genetic search algorithms, requires repeated function (response) evaluations during the search process. A single finite element analysis of large-scale structures requires a significant amount of computational time and resource and hence the need for robust approxima

The present study focuses on the application of artificial neural network (ANN) and Multiple linear Regression (MLR) analysis for developing a model to predict the unconfined compressive strength (UCS) and split tensile strength (STS) of the fiber reinforced clay stabilized with grass ash, fly ash and lime. Unconfined compressive strength and Split tensile strength are the nonlinear functions and becomes difficult for developing a predicting model. Artificial neural networks are the efficient tools for predicting models possessing non linearity and are used in the present study along with regression analysis for predicting both UCS and STS. The data required for the model was obtained by systematic experiments performed on only Kaolin clay, clay mixed with varying percentages of fly ash, grass ash, polypropylene fibers and lime as between 10-20%, 1-4%, 0-1.5% and 0-8% respectively. Further, the optimum values of the various stabilizing materials were determined from the experiments. The effect of stabilization is observed by performing compaction tests, split tensile tests and unconfined compression tests. ANN models are trained using the inputs and targets obtained from the experiments. Performance of ANN and Regression analysis is checked with statistical error of correlation coefficient (R) and both the methods predict the UCS and STS values quite well; but it is observed that ANN can predict both the values of UCS as well as STS simultaneously whereas MLR predicts the values separately. It is also observed that only STS values can be predicted efficiently by MLR.

Neural networks have been successfully used in many applications. Due to their computational complexity, it is difficult to implement them on embedded devices. Neural networks are inherently approximate and thus can be simplified. In this paper, CANNA proposes a gradual training approximation which adaptively sets the level of hardware approximation depending on the neural network's internal error, instead of apply uniform hardware approximation. To accelerate inference, CANNA's layer-based approximation approach selectively relaxes the computation in each layer of neural network, as a function its sensitivity to approximation. For hardware support, we use a configurable floating point unit in Hardware that dynamically identifies inputs which produce the largest approximation error and process them instead in precise mode. We evaluate the accuracy and efficiency of our design by integrating configurable FPUs into AMD's Southern Island GPU architecture. Our experimental evaluation shows that CANNA achieves up to 4.84× (7.13×) energy savings and 3.22× (4.64×) speedup when training four different neural network applications with 0% (2%) quality loss as compared to the implementation on baseline GPU. During the inference phase, our layer-based approach improves the energy efficiency by 4.42× (6.06×) and results in 2.96× (3.98×) speedup while ensuring 0% (2%) quality loss.

In order to test the applicability of support vector regression (SVR) in near-infrared (NIR) spectral analysis of tobacco,NIR calibration model for starch in tobacco were developed by processing NIR spectra and chemically determined starch content data of 187 tobacco samples with SVR,partial least square regression (PLS),multiplicative linear regression (MLR) and error back propagation artificial neural network (BP-ANN). The obtained model was tested through internal validation and external validation with leave-one-out cross validation (LOOCV) and independent sample set. The results showed that the accuracy of SVR model was slightly higher than that of BP-ANN,PLS and MLR models,it implied that SVR algorithm could be applied to NIR analysis of starch in tobacco.

Extreme learning approaches are used to perform sentiment analysis on restaurant evaluations. Sluggish training and overfitting are two problems that traditional supervised learning techniques frequently face. In order to prevent overfitting and speed up training, extreme learning makes use of single hidden layer neural networks with randomly assigned weights. The goal is to create a quick and accurate model for identifying sentiment in meal reviews and to assess the differences between supervised learning techniques and models based on extreme learning. In order to map the scores to attitudes, the study uses a dataset of food reviews, where scores larger than three are interpreted as favourable and scores below that as negative. Training and testing sets are created from the dataset following preparation, which includes handling missing values and choosing pertinent columns. For modeling purposes, text data is transformed into Term Frequency-Inverse Document Frequency (TF-IDF) characteristics. The network is given randomly initialized weights and biases in both single layer and multi layer perceptron implemented. During model training, the loss function is computed, weights and biases are adjusted by backpropagation, and predictions are computed using forward propagation technique. By using a threshold of 0.5, the model's accuracy is assessed. Accuracy scores are used as a statistic for reporting training and testing accuracy. The model further validates the effectiveness for sentiment analysis in the context of food reviews by showing quicker training times and less sensitivity to overfitting. The work presents extreme learning approaches as a competitive substitute for supervised learning, which advances sentiment analysis tools. Comparing the model based on extreme learning to traditional supervised learning methods, experimental results show that the latter achieves competitive accuracy in sentiment analysis. Faster training times and less sensitivity to overfitting are further features. As a strong substitute for supervised learning techniques, this study highlights the effectiveness of extreme learning approaches for sentiment analysis in meal evaluations. Extreme learning improves the efficacy and precision of sentiment analysis models, especially in areas such as restaurant evaluations, furthering the practical uses of sentiment analysis tools.

This study presents establishment of multiple input prediction model for automotive coil spring fatigue life estimation to shorten automotive suspension design process. Automotive suspension design is a lengthy work where any changes of the design lead to repetition of the entire process. It was hypothesised that the established model could be used to predict the spring design fatigue life without using any strain measurements. To initiate this model establishment, five sets of strain and acceleration measurement across different road conditions were collected and used for validations. To include spring stiffness as a parameter, a quarter car model was generated to obtain the force time histories from spring and vertical vibration of vehicle mass. In addition, artificial road profiles of road classes “A” to “D” were also generated for the quarter car simulation. Through adjusting the spring stiffness in the quarter car model, the spring and vehicle responses were varied. The simulated force time histories were used to predict springs’ fatigue life while acceleration time histories were used to calculate ISO 2631 ride-related vertical vibration. Subsequently, multiple linear regression approach was applied to determine the relationship between vehicle body frequency, ISO 2631 ride-related vertical vibration and spring fatigue life. The obtained regression had shown significance to the spring fatigue life with coefficient of determination of 0.8320. Reciprocally, multiple linear regression models were also used to predict the ISO 2631 ride-related vertical vibration with a coefficient of determination at 0.8810 and mean squared error values below 0.3430. To optimise the prediction results, artificial neural network was trained for the fatigue and vibration prediction purposes. The architectures of the artificial neural network were designed in terms of number of neurons and hidden layers to achieve a higher coefficient of determination of 0.9926 and lower mean squared error of 0.0824. For vibration prediction, the vehicle body frequency and spring fatigue life has shown a significant coefficient of determination to the ISO 2631 weighted vertical vibration, reaching 0.9579 with mean squared error of 0.0004. Based on the experimental strain and acceleration results, the predicted fatigue lives of multiple linear regression models were correlated well with the experimental results with coefficient of determination value of 0.9275. Meanwhile, the maximum difference of vibration prediction to experimental value using multiple linear regression models was only 18%. For artificial neural network predictions, the fatigue lives were mostly distributed within 1:2 or 2:1 life correlation and vibration prediction results were within 12%. For a good prediction, the target correlation value was above 0.80 to demonstrate a good fitted curve and the difference below 20%. The trained artificial neural network has shown outstanding capability in fatigue life or ride-related vertical vibration predictions. In this research, the main novelty was the trained artificial neural network for spring fatigue life or ride-related vertical vibration predictions which serve to reduce some procedures of automotive suspension design. The outcome of this study can be used to provide a new knowledge towards the field of fatigue research as well as vehicle ride dynamics. This research contributes to automotive industries especially in suspension spring design where the analysis of fatigue and ride-related vibration are provided.

The modeling and predicting of vegetation Leaf area index (LAI) is an important component of land surface model and assimilation of remote sensing data. The MODIS LAI product (i.e. MOD15A2) is one of the most widely used LAI data sources. However, the time series of MODIS LAI contains some data of low quality. For example, because of the influence of the cloud, aerosol, etc., the MODIS LAI presents the characteristics of the discontinuous in time and space. In fact, the time series of MODIS LAI include both linear and nonlinear components, which cannot be accurately modeled and predicted by either linear method or nonlinear method alone. In this paper, the original LAI time series data were first smoothed with Savitzky-Golay (SG) filtration and linear interpolation; SARIMA, BP neural network and a hybrid method of SARIMA-BP neural network were then used for modeling and predicting MODIS LAI time series. The SARIMA-BP neural network combined both SARIMA and BP neural network, which could model the linear and the nonlinear component of MODIS LAI time series respectively. That is, the final result of SARIMA-BP neural network was the sum of results of the two methods. Experiments showed that the time series of MODIS LAI that were smoothed with the SG filtration and linear interpolation were more smooth than original time series, with a determination coefficient up to 0.981, closer to 1 than that of SARIMA (0.941) and BP neural network (0.884); the correlation coefficient between SARIMA-BP neural network and the observation is 0.991, higher than that of between SARIMA (0.971) or BP neural network (0.942) SARIMA and the observation. Thus, it can be concluded that, the proposed SARIMA-BP neural network method can better adapt to the LAI time series, and it outperforms the SARIMA and BP neural network methods.

This paper presents a complex-valued neuron (CVN) model for real-valued classification problems incorporating a new activation function. The activation function maps complex-valued net-inputs (sum of weighted inputs) of a neuron into bounded real-values, and its role is to divide the net-input space into different regions for different classes. A gradient-descent learning rule has been derived to train the CVN. Such a CVN is able to solve all possible two-input Boolean functions. For further investigation, single layered complex-valued neural networks (Le. without hidden units) are applied on the real-world multi-class classification problems. The results are comparable to the conventional multilayer real-valued neural networks. It is also shown that the performance can be improved further by using their ensembles. Negative correlation learning (NCL) algorithm has been used to create the ensembles. Since NCL is a gradient-descent based algorithm, the proposed activation function is well suited for it.

The Sixth International Symposium on Neural Networks (ISNN 2009)

To develop, train and test neural networks for predicting heart surface potentials (HSPs) from body surface potentials (BSPs). The method re-frames traditional inverse problems of electrocardiography into regression problems, constraining the solution space by decomposing signals with multidimensional Gaussian impulse basis functions. Impulse HSPs were generated with single Gaussian basis functions at discrete heart surface locations and projected to corresponding BSPs using a volume conductor torso model. Both BSP (inputs) and HSP (outputs) were mapped to regular 2D surface meshes and used to train a neural network. Predictive capabilities of the network were tested with unseen synthetic and experimental data. A dense full connected single hidden layer neural network was trained to map body surface impulses to heart surface Gaussian basis functions for reconstructing HSP. Synthetic pulses moving across the heart surface were predicted from the neural network with root mean squared error of 9.1±1.4%. Predicted signals were robust to noise up to 20 dB and errors due to displacement and rotation of the heart within the torso were bounded and predictable. A shift of the heart 40 mm toward the spine resulted in a 4% increase in signal feature localization error. The set of training impulse function data could be reduced, and prediction error remained bounded. Recorded HSPs from in-vitro pig hearts were reliably decomposed using space-time Gaussian basis functions. Activation times calculated from predicted HSPs for left-ventricular pacing had a mean absolute error of 10.4±11.4 ms. Other pacing scenarios were analyzed with similar success. Impulses from Gaussian basis functions are potentially an effective and robust way to train simple neural network data models for reconstructing HSPs from decomposed BSPs. The HSPs predicted by the neural network can be used to generate activation maps that non-invasively identify features of cardiac electrical dysfunction and can guide subsequent treatment options.

Changes on the CIELab values of the dyed materials after the different chemical finishing treatments using artificial neural network (ANN) and linear regression (LR) models have been predicted. The whole structural properties of fabrics and some process data which were from fiber to the finishing parameters were accepted as inputs in these models. The networks having different structures were established, and it was also focus on the parameters which could affect the performance of the established networks. It was determined that we could successfully predict the color differences values occurring on the material after the finishing applications. In addition, we realized that some ANN parameters affected the prediction performance while establishing the models. After training ANN models, the prediction of the color difference values was also tried by linear regression models. Then, extra ANN models were established for all outputs using the parameters as inputs in the LR equations, and the prediction performances of both established models were compared. According to the results, the neural network model gives a more accurate prediction performance than the LR models.

The universal approximation theorem asserts that a single hidden layer neural network approximates continuous functions with any desired precision on compact sets. As an existential result, the universal approximation theorem supports the use of neural networks for various applications, including regression and classification tasks. The universal approximation theorem is not limited to real-valued neural networks but also holds for complex, quaternion, tessarines, and Clifford-valued neural networks. This paper extends the universal approximation theorem for a broad class of hypercomplex-valued neural networks. Precisely, we first introduce the concept of non-degenerate hypercomplex algebra. Complex numbers, quaternions, and tessarines are examples of non-degenerate hypercomplex algebras. Then, we state the universal approximation theorem for hypercomplex-valued neural networks defined on a non-degenerate algebra.KeywordsHypercomplex algebrasNeural networksUniversal approximation theorem