Emulating Complex Dynamical Simulators with Random Fourier Features

In this paper, we investigate the nonlinear, finite dimensional and data independent random Fourier feature expansions that can approximate the popular Gaussian kernel. With recursive least squares algorithm, we develop the Random Fourier Feature Recursive Least Squares algorithm (RFF-RLS), which shows significant performance improvements in simulations when compared with several other online kernel learning algorithms such as Kernel Least Mean Square (KLMS) and Kerne Recursive Least Squares (KRLS). Our results confirm that the RFF-RLS can achieve desirable performance with low computational cost. As for the random Fourier features, the randomization generally results in redundancy. We use an algorithm, namely, Vector Quantization with Information Theoretic Learning (VQIT) to decrease the dictionary size. The resulting sparse dictionary can match the original data distribution well. The RFF-RLS with VQIT can outperform the RFF-RLS without VQIT.

Random Fourier Features (RFF) demonstrate well-appreciated performance in kernel approximation for large-scale situations but restrict kernels to be stationary and positive definite. And for non-stationary kernels, the corresponding RFF could be converted to that for stationary indefinite kernels when the inputs are restricted to the unit sphere. Numerous methods provide accessible ways to approximate stationary but indefinite kernels. However, they are either biased or possess large variance. In this article, we propose the generalized orthogonal random features, an unbiased estimation with lower variance. Experimental results on various datasets and kernels verify that our algorithm achieves lower variance and approximation error compared with the existing kernel approximation methods. With better approximation to the originally selected kernels, improved classification accuracy and regression ability is obtained with our approximation algorithm in the framework of support vector machine and regression.

Kernel based learning is very popular in machine learning but often quite costly with at least quadratic runtime complexity. Random Fourier features and related techniques have been proposed to provide an explicit kernel expansion such that standard techniques with low runtime and memory complexity can be used. This strategy leads to rather high dimensional datasets which is a drawback in many cases. Here, we combine a recently proposed unsupervised selection strategy for random Fourier features [1] with the very efficient supervised relevance learning given by Matrix LVQ. The suggested technique provides reasonable small but very discriminative features sets <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1.</sup>

Advanced Metering Infrastructure (AMI) of the power consumption side, serving as the neural hub of modern power systems, orchestrates real time energy data flows between distributed smart meters and centralized grid control centers, yet this pervasive connectivity introduces critical vulnerabilities to cyber attacks. Traditional intrusion detection systems struggle with the high-dimensional traffic patterns and severe class imbalance inherent in AMI networks, particularly in identifying handful attack variants amidst overwhelming normal traffic. To address these challenges, this paper proposes a detection model incorporating Random Fourier Features (RFF), Adaptive Weighted Focal Loss (AWFL), and a dual-scale convolutional neural network. The model first employs RFF to construct a low-dimensional explicit mapping, replacing the implicit kernel function in kernel PCA. This transforms the original nonlinear eigenvalue problem into linear PCA on random features, thereby reducing computational complexity significantly while preserving the capability of nonlinear dimensionality reduction. Subsequently, AWFL dynamically adjusts the model’s focus on different class samples during training, significantly improving class imbalance handling in network intrusion datasets. Finally, a dual-scale depthwise separable convolution module (DDSCM) is adopted to extract spatial features from input information, enabling more accurate identification of intrusion types. Evaluations on the NSL-KDD dataset and TON_IoT dataset demonstrate that the proposed model achieves superior intrusion detection performance, outperforming various comparative models in multiclass classification tasks regarding accuracy, precision, recall and F1-score.

The complex kernel adaptive filters (CKAFs) developed in complex reproducing kernel Hilbert space (RKHS) improve the performance of complex linear adaptive filters, but result in large burdens of computation and memory. To address these issues, a novel complex random Fourier features mapping (CRFFM) is proposed to approximate the kernel-induced mapping by applying the complexification of real RKHSs, and thus projects the complex-valued data from the original data space into the fixed-dimensional complex random Fourier features space (RFFS). To combat complex-valued non-Gaussian noises, a complex Cauchy loss function is presented, and the complex random Fourier features recursive complex Cauchy (CRFFRCC) algorithm is therefore proposed by the combination of CRFFM and the stochastic recursive method. The proposed CRFFRCC with a linear filter structure can reduce the computational and space complexities of CKAFs, significantly. Monte Carlo simulations conducted in the complex-valued nonlinear channel equalization validate the superiorities of CRFFRCC.

Fault diagnosis plays a vital role in ensuring the operation safety of blast furnaces and improving the quality of molten iron in the ironmaking and steelmaking industry. The blast furnace ironmaking process (BFIP) is intrinsically nonlinear. To address the nonlinearity issue of BFIP, a novel fault diagnosis approach that combines the randomized method, local structure information, and Fisher discriminant analysis is proposed in this paper. Using a randomized feature map, the process data is first mapped onto a randomized explicit low‐dimensional feature space. Compared to kernel methods, explicit low‐dimensional random Fourier features considerably reduce the computational cost, particularly for real‐time fault diagnosis for a large training dataset or a large‐scale process. Additionally, the local structure information contained in the randomized low‐dimensional feature space is extracted. The fault diagnosis performance is improved through the exploration of the local structure of random Fourier features. Finally, the blast furnace iron‐marking process state is determined using Bayesian inference. Case studies on a real‐world BFIP are carried out to demonstrate the superior performance of the proposed method in comparison with other related methods.

Random Fourier feature kernel recursive maximum mixture correntropy algorithm for online time series prediction

This work presents a fast neural surrogate capable of reconstructing fully three-dimensional hemodynamic velocity fields in stenotic and bifurcating microvascular geometries with satisfactory accuracy, avoiding repeated, computationally demanding computational fluid dynamics (CFD) simulations. A Fourier-augmented, coordinate-neural surrogate is presented and assessed for rapid computation of three-dimensional blood-flow fields in a sample geometry. The model is trained on detailed CFD data across a parameter set of stenosis severities that feed a direct mapping from spatial coordinates to velocity components. To mitigate spectral bias and improve accuracy in regions of steep gradients, the input space is embedded with random Fourier features and compared against a conventional multilayer perceptron (MLP) backbone. Predictive ability is assessed upon strict hold-out testing, during which certain arteriolar stenosis cases are excluded from training and treated with the Fourier surrogate. Direct comparison with CFD results reveals that the Fourier MLP achieves nearly CFD fidelity with the coefficient of determination R2 ≥ 0.994 and offers more than 80% reduction in the normalized errors as provided by conventional MLP, with the precise improvement depending on the severity of stenosis. Centerline velocity and cross-sectional profiles further show that the Fourier MLP reconstructs stenosis speed-up and radial profiles more reliably compared to conventional MLP. These results indicate that Fourier feature embedding provides a simple and effective route to robust full-field hemodynamic surrogates for efficient screening of stenosis configurations without resorting to repeated, heavily demanding CFD simulations.

Slim-RFFNet: Slim deep convolution random Fourier feature network for image classification

We live in the era of big data with dataset sizes growing steadily over the past decades. In addition, obtaining expert labels for all the instances is time-consuming and in many cases may not even be possible. This necessitates the development of advanced semi-supervised models that can learn from both labeled and unlabeled data points and also scale at worst linearly with the number of examples. In the context of kernel based semi-supervised models, constructing the training kernel matrix for the large training dataset is expensive and memory inefficient. This paper investigates the scalability of the recently proposed multi-class semi-supervised kernel spectral clustering model (MSSKSC) by means of random Fourier features. The proposed model maps the input data into an explicit low-dimensional feature space. Thanks to the explicit feature maps, one can then solve the MSSKSC optimization formation in the primal, making the complexity of the method linear in number of training data points. The performance of the proposed model is compared with that of recently introduced reduced kernel techniques and Nystrom based MSSKSC approaches. Experimental results demonstrate the scalability, efficiency and faster training computation times of the proposed model over conventional large scale semi-supervised models on large scale real-life datasets.

The clarity of seismic images has a great impact on the results of seismic data interpretation; therefore, high-resolution seismic images can provide finer geological information for seismic exploration. Coordinate-based Multilayer Perceptron(MLP) is easy for gradient-based optimization and machine learning, and represents a more compact order of magnitude than grid sampling. In this paper, we propose a combination of Random Fourier Features(RFF) and coordinate based MLP method to achieve super-resolution reconstruction of seismic images, taking one image coordinate as input and the output is the RGB value of the pixel at that location. Preprocessing the spatial coordinates of the image with Fourier feature mapping enables the MLP to learn higher frequency detail information and achieve super-resolution reconstruction of a single seismic image. Experiments show that the texture and edge details of the final processed seismic image are much clearer.

Convolutional kernel networks based on a convex combination of cosine kernels

Kernel based learning is very popular in machine learning, but many classical methods have at least quadratic runtime complexity. Random fourier features are very effective to approximate shift-invariant kernels by an explicit kernel expansion. This permits to use efficient linear models with much lower runtime complexity. As one key approach to kernelize algorithms with linear models they are successfully used in different methods. However, the number of features needed to approximate the kernel is in general still quite large with substantial memory and runtime costs. Here, we propose a simple test to identify a small set of random fourier features with linear costs, substantially reducing the number of generated features for low rank kernel matrices, while widely keeping the same representation accuracy. We also provide generalization bounds for the proposed approach.

Model-based Reinforcement Learning with Approximate Gaussian Processes using Random Fourier Features

Kernel density estimation (KDE) is one of the most widely used nonparametric density estimation methods. The fact that it is a memory-based method, i.e., it uses the entire training data set for prediction, makes it unsuitable for most current big data applications. Several strategies, such as tree-based or hashing-based estimators, have been proposed to improve the efficiency of the kernel density estimation method. The novel density kernel density estimation method (DMKDE) uses density matrices, a quantum mechanical formalism, and random Fourier features, an explicit kernel approximation, to produce density estimates. This method has its roots in the KDE and can be considered as an approximation method, without its memory-based restriction. In this paper, we systematically evaluate the novel DMKDE algorithm and compare it with other state-of-the-art fast procedures for approximating the kernel density estimation method on different synthetic data sets. Our experimental results show that DMKDE is on par with its competitors for computing density estimates and advantages are shown when performed on high-dimensional data. We have made all the code available as an open source software repository.