Random vector functional link network: Recent developments, applications, and future directions

Random vector functional link (RVFL) network has been successfully employed in diverse domains such as computer vision and machine learning, due to its universal approximation capability. Recently, the shallow RVFL architecture has been extended to deep architectures. In deep architectures, multiple hidden layers are stacked for extracting informative features from the original feature space. Therefore, having rich features, deep models are very successful compared to shallow models. In this article, we propose an extended feature RVFL (efRVFL) model that is trained over extended feature space generated analytically from the original feature space. The proposed efRVFL model has three types of features, i.e., original features, supervised randomized (newly generated) features, and unsupervised randomized features, in its feature matrix. The proposed efRVFL model with additional features has capability to capture nonlinear hidden relationships within the dataset. The proposed efRVFL model is an unstable classifier, and thus, its performance can be improved further via ensemble learning. Ensemble models are stable and accurate and have better generalization performance than single models. Therefore, we also propose an ensemble of extended feature RVFL (en-efRVFL) model. Each base model of en-efRVFL is trained over different feature spaces so that more accurate and diverse base models can be generated. The outcome of the base models is integrated via average voting scheme. Empirical evaluation over <inline-formula> <tex-math notation="LaTeX">$46$</tex-math> </inline-formula> UCI classification datasets demonstrates that the proposed efRVFL and en-efRVFL models have better performance than RVFL and other given deep models. Furthermore, the experimental results over <inline-formula> <tex-math notation="LaTeX">$12$</tex-math> </inline-formula> sparse datasets show that the proposed en-efRVFL model has a winning performance among several deep feedforward neural networks (FNNs).

A novel transient stability assessment (TSA) approach using random vector functional link (RVFL) network optimized by Jaya algorithm, called Jaya-RVFL, is proposed for power systems in this paper. First, by extracting system-level features from phasor measurement unit (PMU) measurements as predictors, an RVFL-based TSA model is proposed. In order to improve the performance of RVFL classifiers, a quantile scaling approach is utilized to optimize the randomization range of input weights via the Jaya algorithm. The simulation results on IEEE 39-bus system and a real-world power system show that the presented method outperforms other popular methods comprising multilayer perception, probabilistic neural network, and support vector machine.

An unsupervised parameter learning model for RVFL neural network

The field of artificial neural networks has a long history of several decades, where the theoretical contributions have progressed with advances in terms of power and memory in present day computers. Some old methods are now rebranded or represented, taking advantage of the power of present day computers. More particularly, we consider the current trend of Random Vector Functional Link Networks, which suggests that the architecture of a system and the learning algorithm should be properly decoupled. In this paper, we evaluate the performance of multi-layers Random Vector Functional Link Network (RVFL)/ extreme machine learning (EML) on four databases of handwritten characters. Particularly, we evaluate the impact of the architecture (number of neurons per hidden layer), and the robustness of the distribution of the results across different runs. By combining the classifier outputs from different runs, we show that such a maximum combination rule provides an accuracy of 95.97% for Arabic digits, 98.03% for Bangla, 98.64% for Devnagari, and 96.30% for Oriya digits. The results confirm that increasing the size of the hidden layers has a significant impact on the accuracy, and allows to reach state-of-the-art performance; however the performance reaches a plateau after a certain size of the hidden layers.

Ship order book forecasting by an ensemble deep parsimonious random vector functional link network

Utilization of Random Vector Functional Link integrated with Marine Predators Algorithm for tensile behavior prediction of dissimilar friction stir welded aluminum alloy joints

Respiratory motion exhibits non-linear and non-stationary behavior in nature and this has been a great hindrance to the accurate prediction of tumor in motion adaptive radiotherapy. Accurate prediction of respiratory motion and subsequent tracking of tumor has been a challenge due to the irregularities and intra-trace variabilities. In order to overcome this issue, prediction models can be trained by using neural networks. However due to the burden of large training data, computational efficacy of existing neural networks can be affected. Moreover, training of neural networks using conventional methods like back-propagation (BP) may result in local minima and it may slow down the learning rate and convergence respectively. As a solution, in this paper, we employed random vector function link (RVFL) based neural networks to train the model in a very efficient way to achieve high accuracy in respiratory motion prediction. In RVFL, the direct link from input features to output layer acts as regularization to prevent the network from overfitting. The proposed method is tested with real respiratory motion traces acquired from 31 patients. Results show that RVFL with the use of direct link performs quite better than without direct link.

Particle size estimate of grinding processes using random vector functional link networks with improved robustness

Distributed learning for Random Vector Functional-Link networks

Parsimonious random vector functional link network for data streams

Alzheimer's disease (AD) is a progressive brain disorder. Machine learning models have been proposed for the diagnosis of AD at early stage. Recently, deep learning architectures have received quite a lot attention. Most of the deep learning architectures suffer from the issues of local minima, slow convergence and sensitivity to learning rate. To overcome these issues, non-iterative learning based deep randomized models especially random vector functional link network (RVFL) with direct links have proven to be successful. However, deep RVFL and its ensemble models are trained only on normal samples. In this paper, deep RVFL and its ensembles are enabled to incorporate privileged information, as the standard RVFL model and its deep models are unable to use privileged information. To fill this gap, we have incorporated learning using privileged information (LUPI) in deep RVFL model, and propose deep RVFL with LUPI framework (dRVFL+). Privileged information is available while training the models. As RVFL is an unstable classifier, we propose ensemble deep RVFL+ with LUPI framework (edRVFL+) which exploits the LUPI as well as the diversity among the base leaners for better classification. Unlike traditional ensemble approach wherein multiple base learners are trained, the proposed edRVFL+ model optimises a single network and generates an ensemble via optimization at different levels of random projections of the data. Both dRVFL+ and edRVFL+ efficiently utilise the privileged information which results in better generalization performance. In LUPI framework, half of the available features are used as normal features and rest as the privileged features. However, we propose a novel approach for generating the privileged information. We utilise different activation functions while processing the normal and privileged information in the proposed deep architectures. To the best of our knowledge, this is first time that a separate privileged information is generated. The proposed dRVFL+ and edRVFL+ models are employed for the diagnosis of Alzheimer's disease. Experimental results demonstrate the superiority of the proposed dRVFL+ and edRVFL+ models over baseline models. Thus, the proposed edRVFL+ model can be utilised in clinical setting for the diagnosis of AD.

Convolutional sparse coding-based deep random vector functional link network for distress classification of road structures

Randomized feed-forward artificial neural networks (ANNs) have been employed in various domains. This paper was written in order to assess the efficiency of the basic forms of randomized feed-forward ANNs, which are randomized weight artificial neural network, random vector functional link network, extreme learning machine, and radial bases function neural network. In order to compare these methods, a complex dataset, which is the power load of a small house dataset, was used. Obtained results showed that lower training error rates were achieved by randomized vector functional link network. On the other hand, lower test error rates were achieved by ELM. Furthermore, ELM has faster training and test stages than the other employed randomized ANNs.

Randomized neural network (RNN) is a highly feasible solution in the era of big data because it offers a simple and fast working principle in processing dynamic and evolving data streams. This paper proposes a novel RNN, namely recurrent type-2 random vector functional link network (RT2McRVFLN), which provides a highly scalable solution for data streams in a strictly online and integrated framework. It is built upon the psychologically inspired concept of metacognitive learning, which covers three basic components of human learning: what-to-learn, how-to-learn, and when-to-learn. The what-to-learn selects important samples on the fly with the use of online active learning scenario, which renders our algorithm an online semi-supervised algorithm. The how-to-learn process combines an open structure of evolving concept and a randomized learning algorithm of random vector functional link network (RVFLN). The efficacy of the RT2McRVFLN has been numerically validated through two real-world case studies and comparisons with its counterparts, which arrive at a conclusive finding that our algorithm delivers a tradeoff between accuracy and simplicity.

Background: QoS parameters are volatile in nature and possess high nonlinearity, thus making the IoT-based service and recommendation process challenging. Methods: An efficient and accurate forecasting model is lacking in this area and needs to be explored. Though an artificial neural network is a prominent option for capturing such nonlinearities, its efficiency is limited by the structural complexity and iterative learning method. The random vector functional link network (RVFLN) significantly reduces the time complexity by randomly assigning input weights and biases without further modification. Only output layer weights are calculated iteratively by gradient methods or non-iteratively by least square methods. It is an efficient algorithm with low time complexity and can handle complex domain problems without compromising accuracy. Motivated by these characteristics, this article develops an RVFLN-based model for forecasting QoS parameter sequences. Results: Two real-world IoT-enabled web service dataset series are used in developing and evaluating the effectiveness of RVFLN-based forecasts in terms of three performance metrics. Conclusion: Experimental results, comparative studies, and statistical tests are conducted to establish the superiority of the proposed approach over four other similar forecasting techniques.