Training of an Extreme Learning Machine Autoencoder Based on an Iterative Shrinkage-Thresholding Optimization Algorithm

Various deconvolution algorithms for acoustic source are developed to improve spatial resolution and suppress sidelobe of the conventional beamforming. To improve the computational efficiency and solution convergence of deconvolution, this paper proposes a Fourier-based improved fast iterative shrinkage thresholding algorithm. Simulations and experiments show that Fourier-based improved fast iterative shrinkage thresholding algorithm can achieve excellent acoustic identification performance, with high computational efficiency and good convergence. For Fourier-based improved fast iterative shrinkage thresholding algorithm, the larger the weight coefficient, the narrower the mainlobe width, and the better the convergence, but the spurious source also increases. The recommended weight coefficient for the array described herein is 3. In addition, like other Fourier-based deconvolution algorithms, Fourier-based improved fast iterative shrinkage thresholding algorithm using irregular focus grid can obtain better acoustic source identification performance than using the conventional regular focus grid.

Magnetotelluric (MT) sounding data are easily contaminated by various noise sources, especially noise with a long duration (even full-time noise), which makes it difficult to obtain accurate values when calculating the weight factors of the response function, resulting in distortion of the response results. Based on blind source separation theory, fast independent component analysis (FastICA) can separate this kind of noise. However, this method is challenged by the number of separated field sources and the unequal signal amplitude before and after processing. We have developed a novel signal noise separation method, improving the traditional FastICA, where a correlation coefficient is used to compute the number of field sources, and the fast iterative shrinkage threshold algorithm (FISTA) is used to adjust the signal amplitude problem before and after FastICA decomposition. Compared with common field source division methods and the traditional FastICA, the experimental results indicate that our method can separate and remove noise with a long duration, increase the signal-to-noise ratio of the data, and improve the MT response curves. Meanwhile, case studies of measured data illustrate that our method obtains a more robust magnetotelluric response than the conventional robust method and FastICA.

Accelerated parabolic Radon domain 2D adaptive multiple subtraction with fast iterative shrinkage thresholding algorithm and its application in parabolic Radon domain hybrid demultiple method

After multiple prediction, adaptive multiple subtraction is essential for the success of multiple removal. The 3D blind separation of convolved mixtures (3D BSCM) method, which is effective in conducting adaptive multiple subtraction, needs to solve an optimization problem containing L1-norm minimization constraints on primaries by the iterative reweighted least-squares (IRLS) algorithm. The 3D BSCM method can better separate primaries and multiples than the 1D/2D BSCM method and the method with energy minimization constraints on primaries. However, the 3D BSCM method has high computational cost because the IRLS algorithm achieves nonquadratic optimization with an LS optimization problem solved in each iteration. In general, it is good to have a faster 3D BSCM method. To improve the adaptability of field data processing, the fast iterative shrinkage thresholding algorithm (FISTA) is introduced into the 3D BSCM method. The proximity operator of FISTA can solve the L1-norm minimization problem efficiently. We demonstrate that our FISTA-based 3D BSCM method achieves similar accuracy of estimating primaries as that of the reference IRLS-based 3D BSCM method. Furthermore, our FISTA-based 3D BSCM method reduces computation time by approximately 60% compared with the reference IRLS-based 3D BSCM method in the synthetic and field data examples.

<sec>An algorithm of reconstructing phaseless radiation source based on singular value decomposition (SVD) regularization and fast iterative shrinkage-thresholding algorithm (FISTA) is proposed in this work, aiming at efficiently identifying electromagnetic interference (EMI) sources in integrated circuits (ICs). The method acquires electromagnetic field data through near-field scanning and reconstructs an equivalent dipole array on the surface of the radiation source by using the source reconstruction method (SRM). In the reconstruction process, the SVD regularization term enhances the algorithm's stability and noise resistance, while the FISTA accelerates the convergence speed.</sec><sec>In order to validate the effectiveness of the proposed method, dipole array reconstruction is first performed using near-field data at a height of 5 mm for a patch antenna simulation model, followed by analyzing the magnetic field data at a 10 mm validation plane. At the 35th iteration, the total relative error of the reconstruction is 1.21%. The influence of the regularization parameter <i>α</i> on the result is then investigated, and it is found that when <i>α</i> = 0.05 the error is minimized. The method is also tested under different Gaussian white noise conditions, and the relative error is kept below 5%, which demonstrates strong robustness.</sec><sec>Finally, the experiments on chips are conducted to verify the method. The proposed method converges stably within 35 iterations, with a relative error of 2.3% in the reconstruction results. The proposed method reduces the total iteration time to 61.7% of the single-layer phaseless interpolation algorithm, while achieving a 52% lower relative error than the double-layer phasless iteration algorithm. The experimental results show that the proposed method can reconstruct phaseless radiation source efficiently and accurately, and has good noise robustness, which is suitable for EMI analysis in ICs.</sec>

Regularization technology is an efficient and robust method to implement forward-looking imaging of scanning radar, and the selection of regularization parameter is quite important for the imaging performance. When the imaging scene contains sparse strong targets as well as a weak background, different regularization parameters should be adopted. A dual-channel fast iterative shrinkage-thresholding (D-FIST) regularization algorithm is proposed. First, by setting a threshold with mean segmentation method, the strong targets are extracted from the echo to formulate the data of channel I, and the entire data are included in channel II. The L-curve method is adopted to determine the regularization parameter for two channels data, respectively. Then, the data are processed by the FIST algorithm with corresponding parameters. At last, the imaging result is obtained by combining the results of two channels. Simulation experiments show that the imaging results of D-FIST are better than that of direct method, especially in a low SNR situation.

Spaceborne airborne bistatic linear array synthetic aperture radar (SA-BiLASAR) is a novel type of bistatic SAR (BiSAR) with multi-view three-dimensional (3-D) imaging capabilities. However, due to the separation of transceiver platforms and the structure of linear array antenna, the form of its motion errors is variable. Hence the estimation and compensation of its motion errors are more difficult than traditional SAR system. In this paper, an autofocus method is proposed for SA-BiLASAR 3-D sparse imaging based on maximum sharpness estimation (MSE). Firstly, the sparse imaging model of SA-BiLASAR and the spatial-variant characteristic of its motion errors are discussed. Secondly, combined with the MSE principle, the reconstruct procedure of SA-BiLASAR is presented via fast iterative shrinkage thresholding algorithm (FISTA). Lastly, the complex motion errors of SA-BiLASAR are estimated and compensated by the proposed method. Simulation results demonstrate the effectiveness of the sparse autofocus approach in the case of the different types of motion errors.

Statistical iterative reconstruction in Cone-beam computed tomography (CBCT) uses prior knowledge to form different kinds of regularization terms. The total variation (TV) regularization has shown state-of-the-art performance in suppressing noises and preserving edges. However, it produces the well-known staircase effect. In this paper, a method that involves second-order differential operators was employed to avoid the staircase effect. The ability to avoid staircase effect lies in that higher-order derivatives can avoid over-sharpening the regions of smooth intensity transitions. Meanwhile, a fast iterative shrinkage-thresholding algorithm was used for the corresponding optimization problem. The proposed Hessian Schatten norm-based regularization keeps lots of favorable properties of TV, such as translation and scale invariant, with getting rid of the staircase effect that appears in TV-based reconstructions. The experiments demonstrated the outstanding ability of the proposed algorithm over TV method especially in suppressing the staircase effect.

The “fast iterative shrinkage-thresholding algorithm,” a.k.a. FISTA, is one of the most well known first-order optimization scheme in the literature, as it achieves the worst-case $O(1/k^2)$ optimal convergence rate for objective function value. However, despite such an optimal theoretical convergence rate, in practice the (local) oscillatory behavior of FISTA often damps its efficiency. Over the past years, various efforts have been made in the literature to improve the practical performance of FISTA, such as monotone FISTA, restarting FISTA, and backtracking strategies. In this paper, we propose a simple yet effective modification to the original FISTA scheme which has two advantages: It allows us to (1) prove the convergence of generated sequence and (2) design a so-called lazy-start strategy which can be up to an order faster than the original scheme. Moreover, we propose novel adaptive and greedy strategies which further improve the practical performance. The advantages of the proposed schemes are tested through problems arising from inverse problems, machine learning, and signal/image processing.

The construction of brain functional network based on resting-state functional magnetic resonance imaging (fMRI) is an effective method to reveal the mechanism of human brain operation, but the common brain functional network generally contains a lot of noise, which leads to wrong analysis results. In this paper, the least absolute shrinkage and selection operator (LASSO) model in compressed sensing is used to reconstruct the brain functional network. This model uses the sparsity of L1-norm penalty term to avoid over fitting problem. Then, it is solved by the fast iterative shrinkage-thresholding algorithm (FISTA), which updates the variables through a shrinkage threshold operation in each iteration to converge to the global optimal solution. The experimental results show that compared with other methods, this method can improve the accuracy of noise reduction and reconstruction of brain functional network to more than 98%, effectively suppress the noise, and help to better explore the function of human brain in noisy environment.

In this work, a new adaptive extreme learning machine (ELM) neural network-based fuzzy controller is designed and simulated for implementing speed regulation in a permanent magnet synchronous motor. ELM is a neural model wherein the number of hidden neurons to be placed in the hidden layer is tuned during the process of neural network training itself. A new adaptive ELM model is developed for placing the number of hidden neurons in the hidden layer, and this new adaptive ELM is tuned with artificial bee colony (ABC) algorithm for optimizing its weight parameters and also the number of hidden neurons. Fuzzy proportional–integral (PI) controller is developed in this work in order to eliminate the steady-state error. The new adaptive ELM neural model optimized with ABC algorithm is applied to tune the input parameters of the fuzzy PI controller and also on optimizing the rules and fuzzy membership functions. The optimized adaptive ELM neural network-based fuzzy PI controller is utilized to investigate the speed regulation of permanent magnet synchronous motor (PMSM) in this work. The developed new PI controller with the PMSM is tested for its performance characteristics and to prove its validity is compared with the traditional controller and other heuristic controllers proposed in earlier literature works.

The reconstruction of cone-beam x-ray luminescence computed tomography (CB-XLCT) is an ill-posed inversion problem because of incomplete data and lack of prior information. To improve the illness of the inversion problem, the data fidelity and regularization term are two key aspects for the reconstruction model. However, there is not much research considering the statistical characteristics of data in XLCT reconstruction, although many various regularizations are studied. To make full use of the data noise model, a strategy combing the maximum likelihood expectation estimation (MLEM) algorithm and the regularization-type algorithm is proposed. In the MLEM algorithm, the Poisson noise is considered for accurate data model. The result by the regularization-type algorithm is used as the specific initial image for the MLEM to improve the reconstruction quality and convergence speed of the MLEM. There are two main steps in the proposed strategy. Firstly, the fast iterative shrinkage-thresholding algorithm (FISTA) with a large regularization parameter is used to get the sparse solution quickly. Secondly, the sparse solution is used as the initial iteration value of the MLEM. The proposed algorithm is named as FISTA-MLEM. Through the stepwise strategy, the image sparsity is guaranteed and the accuracy of the reconstruction is maintained. Result of phantom experiment shows the FISTA-MLEM method presents better contrast to noise ratio and shape similarity compared with other traditional methods, such as ART, Tikhonov, FISTA and TSVD.

Image reconstruction for Electrical Resistance Tomography (ERT) is an ill-posed nonlinear inverse problem. Considering the influence of the sparse measurement data on the quality of the reconstructed image, the l1-regularized least-squares program (l1 regularized LSP) is introduced to solve the inverse problem in this paper. To meet the need of high speed in ERT, the fast iterative shrinkage-thresholding algorithm (FISTA) is employed for image reconstruction in our work. Simulation results of the FISTA and l1_ls algorithm show that the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularized LSP is superior to the l2 regularization method, especially in avoiding the over-smoothing of the reconstructed image. In addition, to improve the convergence speed and imaging quality in FISTA algorithm, the initial guess is calculated with the conjugate gradient method. Comparative simulation results demonstrate the feasibility of FISTA in ERT system and its advantage over the l1_ls regularization method.

Compressed sensing is a technique to recover a sparse vector from its underdetermined linear measurements. Since a naive <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{0}$</tex> optimization approach is hard to tackle due to the discreteness and the non-convexity of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{0}$</tex> norm, a relaxed problem of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}-\ell_{2}$</tex> optimization is often employed for the reconstruction of the sparse vector especially when the measurement noise is not negligible. FISTA (fast iterative shrinkage-thresholding algorithm) is one of popular algorithms for the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}-\ell_{2}$</tex> optimization, and is known to achieve optimal convergence rate among the first order methods. Recently, the employment of optical circuits for various signal processing including deep neural networks has been considered intensively, but it is difficult to implement FISTA with the optical circuit, because it requires operations of divisions with a dynamic value in the algorithm. In this paper, assuming the implementation with the optical circuit, we propose an ADMM (alternating direction method of multipliers) based algorithm for the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}-\ell_{2}$</tex> optimization. It is true that an ADMM based algorithm for the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}-\ell_{2}$</tex> optimization has been already proposed in the literature, but the proposed algorithm is derived with the different formulation from the existing method, and unlike the existing ADMM based algorithm, the proposed algorithm does not include the calculation of the inverse of a matrix. Computer simulation results demonstrate that the proposed algorithm can achieve comparable performance as FISTA or existing ADMM based algorithm while requiring no division operations and no matrix inversions.

In this paper, we have proposed a new algorithm to train neural network, called Class-Constrained Extreme Learning Machine (C\(^{2}\)ELM), which is based on Extreme Learning Machine (ELM). In C\(^{2}\)ELM, we use class information to constrain different parts of connection weights between input layer and hidden layer using Extreme Learning Machine Auto Encoder (ELM-AE). In this way, we add class information to the connection weights and make the features in the hidden layer which are learned from input space be more discriminative than other methods based on ELM. Meanwhile, C\(^{2}\)ELM can retain the advantages of ELM. The experiments shown that C\(^{2}\)ELM is effective and efficient and can achieve a higher performance in contrast to other ELM based methods.