Hyper-Parameter-Free Latent Factor Analysis via Adaptive Moment Estimation-Incorporated Particle Swarm Optimization

User-side Quality-of-Service (QoS) data are vital for efficient cloud service selection, while available QoS data in real applications are commonly described by a high-dimensional and Sparse (HiDS) matrix due to a) increasing users and services, and b) the impossibility of observing the full invoking mapping among users and services. A latent factor analysis (LFA) model has proven to be efficient in performing representation learning on such an HiDS QoS matrix. However, existing LFA models commonly adopt a first-order optimizer that cannot makes it approach the second-order stationary point of the learning objective, thereby resulting in accuracy loss. Aiming at addressing this issue, this study proposes to incorporate a Generalized Nesterov's Acceleration (GNA) method in to a Hessian-vector algorithm for LFA, thereby establishing a GNA-incorporated Hessian-vector-based LFA (GNHL) model with two-fold ideas: a) adopting the principle of a Hessian-vector method to acquire a proper Newton step efficiently, and b) incorporating a GNA method into its linear search for accelerating its convergence rate. Experimental results on six real QoS datasets demonstrate that a GNHL model outperforms state-of-the-art LFA models in generating highly accurate predictions for missing QoS data with low computational burden.

The latent factor analysis (LFA) model has been widely used to uncover latent relationships from high-dimensional sparse (HiDS) matrices. However, the performance of LFA depends largely on the hyper-parameter value used in the model training. Traditional hyper-parameter tuning methods such as grid search suffer from inefficiency and inaccuracy. In recent years, the particle swarm optimization (PSO) algorithm offers an intelligent approach to adaptively adjust the hyper-parameter of LFA. However, the global optimal solution of the hyper-parameter tuning problem is not fixed due to its dynamic decision space. Therefore, it is difficult for PSO to determine the best hyper-parameter for each training iteration. To address this problem, this paper proposes a novel hyper-parameter adaptive adjustment algorithm called dynamic stochastic reorientation PSO (DSR-PSO) that adapts to constantly changing decision spaces. By randomly adjusting the search directions of particles and perturbing the elite particles, the dynamic property of the DSR-PSO can be enhanced, so that the hyper-parameter can be adjusted in real time throughout the model training process. Furthermore, this paper proves the convergence of the DSR-PSO and gives its convergence condition by discussing the distribution of the characteristic roots. Finally, this paper proposes the DSR-PSO-based LFA (DPL) model by incorporating the DSR-PSO-based hyper-parameter adjustment into the LFA to promote its model training, and analyzes its complexity. Experimental results on benchmark datasets show that the proposed DPL surpasses state-of-the-art LFA models in terms of accuracy and efficiency.

Latent factor analysis (LFA) is highly efficient in knowledge discovery from high-dimensional and sparse (HiDS) matrices frequently encountered in big data and web service related applications. A stochastic gradient descent (SGD) algorithm is commonly adopted as a learning algorithm for LFA owing to its high efficiency. However, its sequential nature makes it less scalable when processing large-scale data. Although an alternating SGD algorithm decouples an LFA process to achieve parallelization, its performance relies on its hyper-parameter selection that is highly expensive to tune. To address it, this paper presents three adaptive alternating SGD algorithms, thus leading to three Parallel Adaptive LFA (PAL) models for LFA on large-scale HiDS matrices. Experimental studies on HiDS matrices from industrial service applications show that the proposed PAL models perform significantly better than existing ones in terms of both convergence rate and computational efficiency, as well as achieve competitive prediction accuracy for missing data.

In this article we describe a novel Adaptive particle swarm optimization (APSO) algorithm based on population diversity information. It is presented to solve the precocious convergence problem of particle swarm optimization algorithm. The APSO algorithm uses the information of the population diversity to adjust nonlinearly inertia weight. Velocity mutation factor and position interchange factor are both introduced and the global performance is clearly improved. The APSO algorithm is applied to optimization of parameters in the optimal model based on support vector machine (SVM). SVM is a popular classification method with many diverse applications. A novel Adaptive particle swarm optimization (APSO) based approach for parameter determination and feature selection of the SVM, termed APSO+SVM is developed. The illustrating example shows that the classification accuracy of APSO+SVM is higher than other traditional methods of classification, so using APSO+SVM method to classify is feasible and effective.

Aiming at the control variables of reactive power optimization are discrete, and some parameters in the standard particle swarm optimization (PSO) algorithm need to be predefined by test, so the algorithm's practicability is restricted. For these reasons, an adaptive particle swarm optimization (APSO) algorithm is proposed by the authors. APSO introduces the self-adaptive tuning strategy and boundary constraint conditions can find the global optimal solution and solve the discrete variables. The reactive power optimization results of the standard IEEE-30-bus power system show that APSO is efficient than standard PSO. The global convergence accuracy and convergence stability is obviously improved compared with that of PSO.

A recommender system (RS) relying on latent factor analysis usually adopts stochastic gradient descent (SGD) as its learning algorithm. However, owing to its serial mechanism, an SGD algorithm suffers from low efficiency and scalability when handling large-scale industrial problems. Aiming at addressing this issue, this study proposes a momentum-incorporated parallel stochastic gradient descent (MPSGD) algorithm, whose main idea is two-fold: a) implementing parallelization via a novel data-splitting strategy, and b) accelerating convergence rate by integrating momentum effects into its training process. With it, an MPSGD-based latent factor (MLF) model is achieved, which is capable of performing efficient and high-quality recommendations. Experimental results on four high-dimensional and sparse matrices generated by industrial RS indicate that owing to an MPSGD algorithm, an MLF model outperforms the existing state-of-the-art ones in both computational efficiency and scalability.

How to extract useful knowledge from a high-dimensional and sparse (HiDS) matrix efficiently is critical for many big data-related applications. A latent factor (LF) model has been widely adopted to address this problem. It commonly relies on an iterative learning algorithm like stochastic gradient descent. However, an algorithm of this kind commonly consumes many iterations to converge, resulting in considerable time cost on large-scale datasets. How to accelerate an LF model's training process without accuracy loss becomes a vital issue. To address it, this study innovatively proposes a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</u> ultilayered-and-randomized <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</u> atent <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</u> actor (MLF) model. Its main idea is two-fold: a) adopting randomized-learning to train LFs for implementing a ‘one-iteration’ training process for saving time; and 2) adopting the principle of a generally multilayered structure as in a deep forest or multilayered extreme learning machine to structure its LFs, thereby enhancing its representative learning ability. Empirical studies on six HiDS matrices from real applications demonstrate that compared with state-of-the-art LF models, an MLF model achieves significantly higher computational efficiency with satisfactory prediction accuracy. It has the potential to handle LF analysis on a large scale HiDS matrix with real-time requirements.

Compensation for target speckle phase by use of the combination of the adaptive particle swarm optimization algorithm and the array detector method in heterodyne detection

An adaptive Particle Swarm Optimization algorithm for fire source identification of the utility tunnel fire

High-dimensional and sparse (HiDS) matrices are frequently adopted to describe the complex relationships in various big data-related systems and applications. A Position-transitional Latent Factor Analysis (PLFA) model can accurately and efficiently represent an HiDS matrix. However, its involved latent factors are optimized by particle-swarm-optimization-incorporated stochastic gradient descent with the specific gradient direction step-by-step, which may cause a suboptimal solution. To address this issue, this paper proposes a <u>S</u>equential-<u>G</u>roup-<u>D</u>ifferential-<u>E</u>volution (SGDE) algorithm to refine the latent factors optimized by a PLFA model, thereby achieving a highly-accurate SGDE-PLFA model to HiDS matrices. The main idea of SGDE-PLFA is two-fold. First, it divides the obtained latent factors by PLFA into row and column sub-groups to ensure the efficient low-dimensional optimization. Second, it initializes and optimizes these sub-group latent factors with our improved Differential Evolution algorithm to search more accurate solutions nearby the suboptimum. As demonstrated by the extensive experiments on four HiDS matrices, an SGDE-PLFA model outperforms the state-of-the-art models in representing HiDS matrices.

Stochastic differential equation (SDE) models are the foundation for pricing and hedging financial derivatives. The drift and volatility functions in SDE models are typically chosen to be algebraic functions with a small number ( < 5 ) of parameters which can be calibrated to market data. A more flexible approach is to use neural networks to model the drift and volatility functions, which provides more degrees-of-freedom to match observed market data. Training of models requires optimizing over an SDE, which is computationally challenging. For European options, we develop a fast stochastic gradient descent (SGD) algorithm for training the neural network-SDE model. Our SGD algorithm uses two independent SDE paths to obtain an unbiased estimate of the direction of steepest descent. For American options, we optimize over the corresponding Kolmogorov partial differential equation (PDE). The neural network appears as coefficient functions in the PDE. Models are trained on large datasets (many contracts), requiring either large simulations (many Monte Carlo samples for the stock price paths) or large numbers of PDEs (a PDE must be solved for each contract). Numerical results are presented for real market data including S&P 500 index options, S&P 100 index options, and single-stock American options. The neural-network-based SDE models are compared against the Black-Scholes model, the Dupire's local volatility model, and the Heston model. Models are evaluated in terms of how accurate they are at pricing out-of-sample financial derivatives, which is a core task in derivative pricing at financial institutions. Specifically, we calibrate a neural network-SDE model to market data for a financial derivative on an asset with price S t with a payoff function g ( s ) , and we then evaluate its generalization accuracy for a financial derivative on the same asset S t but with a different payoff function f ( s ) . In addition to comparing out-of-sample pricing accuracy, we evaluate the hedging performance of the neural network-SDE model.

Cable-driven parallel robots (CDPRs) have been widely used in engineering fields because of their significant advantages including high load-bearing capacity, large workspace, and low inertia. However, the impact of convergence speed and solution accuracy of optimization approaches on optimal performances can become a key issue when it comes to the optimal design of CDPR applied to large storage space. An adaptive adjustment inertia weight particle swarm optimization (AAIWPSO) algorithm is proposed for the multi-objective optimal design of CDPR. The kinematic and static models of CDPR are established based on the principle of virtual work. Subsequently, two performance indices including workspace and dexterity are derived. A multi-objective optimization model is established based on performance indices. The AAIWPSO algorithm introduces an adaptive adjustment inertia weight to improve the convergence efficiency and accuracy of traditional particle swarm optimization (PSO) algorithm. Numerical examples demonstrate that final convergence values of the objective function by the AAIWPSO algorithm can almost be 14∼20% and 19∼40% higher than those by the PSO algorithm and genetic algorithm (GA) for the optimal design of CDPR with different configurations and masses of end-effectors, respectively.

This paper considers the problem of finding the position of a passive target using noisy time difference of arrival (TDOA) measurements, obtained from multiple transmitters and a single receiver. The maximum likelihood (ML) estimator’s objective function is extremely nonlinear and non-convex, making it impossible to use traditional optimization techniques. In this regard, this paper proposes the chaos-enhanced adaptive hybrid butterfly particle swarm optimization algorithm, named CAHBPSO, as the hybridization of butterfly optimization (BOA) and particle swarm optimization (PSO) algorithms, to estimate passive target position. In the proposed algorithm, an adaptive strategy is employed to update the sensory fragrance of BOA algorithm, and chaos theory is incorporated into the inertia weight of PSO algorithm. Furthermore, an adaptive switch probability is employed to combine global and local search phases of BOA with the PSO algorithm. Additionally, the semidefinite programming is employed to convert the considered problem into a convex one. The statistical comparison on CEC2014 benchmark problems shows that the proposed algorithm provides a better performance compared to well-known algorithms. The CAHBPSO method surpasses the BOA, PSO and semidefinite programming (SDP) algorithms for a broad spectrum of noise, according to simulation findings, and achieves the Cramer–Rao lower bound (CRLB).

Energy optimization strategy of vehicle DCS system based on APSO algorithm

This paper presents a novel optimal H&infin; tracking-based adaptive fuzzy neural network controller (HAFNNC) for a variable speed wind turbine. The main objective of the controller is to optimise the energy captured from the wind. In the presence of large uncertainties, H&infin; control approach produces oscillatory phenomenon due to the higher needed gain. In order to reduce this gain, fuzzy neural network (FNN) with online adaptation of the parameters is used to estimate the uncertain parts of the system plant and hence enable a lower gain to be used. To eliminate the trade-offs between the H&infin; tracking performance and the high gain at the control input, we have introduced a new method based on the modification of the output tracking error through the use of both the exponential function and the adaptive particle swarm optimisation (APSO) algorithm. The stability and effectiveness of the proposed method are proved by Lyapunov method and the simulations are given to demonstrate the performance of the proposed approach.