Improving “Fast Iterative Shrinkage-Thresholding Algorithm”: Faster, Smarter, and Greedier

Various deconvolution algorithms for acoustic source are developed to improve spatial resolution and suppress sidelobe of the conventional beamforming. To improve the computational efficiency and solution convergence of deconvolution, this paper proposes a Fourier-based improved fast iterative shrinkage thresholding algorithm. Simulations and experiments show that Fourier-based improved fast iterative shrinkage thresholding algorithm can achieve excellent acoustic identification performance, with high computational efficiency and good convergence. For Fourier-based improved fast iterative shrinkage thresholding algorithm, the larger the weight coefficient, the narrower the mainlobe width, and the better the convergence, but the spurious source also increases. The recommended weight coefficient for the array described herein is 3. In addition, like other Fourier-based deconvolution algorithms, Fourier-based improved fast iterative shrinkage thresholding algorithm using irregular focus grid can obtain better acoustic source identification performance than using the conventional regular focus grid.

Orthogonal transformations, proper decomposition, and the Moore–Penrose inverse are traditional methods of obtaining the output layer weights for an extreme learning machine autoencoder. However, an increase in the number of hidden neurons causes higher convergence times and computational complexity, whereas the generalization capability is low when the number of neurons is small. One way to address this issue is to use the fast iterative shrinkage-thresholding algorithm (FISTA) to minimize the output weights of the extreme learning machine. In this work, we aim to improve the convergence speed of FISTA by using two fast algorithms of the shrinkage-thresholding class, called greedy FISTA (G-FISTA) and linearly convergent FISTA (LC-FISTA). Our method is an exciting proposal for decision-making involving the resolution of many application problems, especially those requiring longer computational times. In our experiments, we adopt six public datasets that are frequently used in machine learning: MNIST, NORB, CIFAR10, UMist, Caltech256, and Stanford Cars. We apply several metrics to evaluate the performance of our method, and the object of comparison is the FISTA algorithm due to its popularity for neural network training. The experimental results show that G-FISTA and LC-FISTA achieve higher convergence speeds in the autoencoder training process; for example, in the Stanford Cars dataset, G-FISTA and LC-FISTA are faster than FISTA by 48.42% and 47.32%, respectively. Overall, all three algorithms maintain good values of the performance metrics on all databases.

First-order methods are used widely for large scale optimization problems in signal/image processing and machine learning, because their computation depends mildly on the problem dimension. Nesterov's fast gradient method (FGM) has the optimal convergence rate among first-order methods for smooth convex minimization; its extension to non-smooth case, the fast iterative shrinkage-thresholding algorithm (FISTA), also satisfies the optimal rate; thus both algorithms have gained great interest. We recently introduced a new optimized gradient method (OGM) (for smooth convex functions) having a theoretical convergence speed that is 2× faster than Nesterov's FGM. This paper further discusses the convergence analysis of OGM and explores its fast convergence on an image restoration problem using a smoothed total variation (TV) regularizer. In addition, we empirically investigate the extension of OGM to nonsmooth convex minimization for image restoration with l 1 -sparsity regularization.

Magnetotelluric (MT) sounding data are easily contaminated by various noise sources, especially noise with a long duration (even full-time noise), which makes it difficult to obtain accurate values when calculating the weight factors of the response function, resulting in distortion of the response results. Based on blind source separation theory, fast independent component analysis (FastICA) can separate this kind of noise. However, this method is challenged by the number of separated field sources and the unequal signal amplitude before and after processing. We have developed a novel signal noise separation method, improving the traditional FastICA, where a correlation coefficient is used to compute the number of field sources, and the fast iterative shrinkage threshold algorithm (FISTA) is used to adjust the signal amplitude problem before and after FastICA decomposition. Compared with common field source division methods and the traditional FastICA, the experimental results indicate that our method can separate and remove noise with a long duration, increase the signal-to-noise ratio of the data, and improve the MT response curves. Meanwhile, case studies of measured data illustrate that our method obtains a more robust magnetotelluric response than the conventional robust method and FastICA.

Accelerated parabolic Radon domain 2D adaptive multiple subtraction with fast iterative shrinkage thresholding algorithm and its application in parabolic Radon domain hybrid demultiple method

After multiple prediction, adaptive multiple subtraction is essential for the success of multiple removal. The 3D blind separation of convolved mixtures (3D BSCM) method, which is effective in conducting adaptive multiple subtraction, needs to solve an optimization problem containing L1-norm minimization constraints on primaries by the iterative reweighted least-squares (IRLS) algorithm. The 3D BSCM method can better separate primaries and multiples than the 1D/2D BSCM method and the method with energy minimization constraints on primaries. However, the 3D BSCM method has high computational cost because the IRLS algorithm achieves nonquadratic optimization with an LS optimization problem solved in each iteration. In general, it is good to have a faster 3D BSCM method. To improve the adaptability of field data processing, the fast iterative shrinkage thresholding algorithm (FISTA) is introduced into the 3D BSCM method. The proximity operator of FISTA can solve the L1-norm minimization problem efficiently. We demonstrate that our FISTA-based 3D BSCM method achieves similar accuracy of estimating primaries as that of the reference IRLS-based 3D BSCM method. Furthermore, our FISTA-based 3D BSCM method reduces computation time by approximately 60% compared with the reference IRLS-based 3D BSCM method in the synthetic and field data examples.

<sec>An algorithm of reconstructing phaseless radiation source based on singular value decomposition (SVD) regularization and fast iterative shrinkage-thresholding algorithm (FISTA) is proposed in this work, aiming at efficiently identifying electromagnetic interference (EMI) sources in integrated circuits (ICs). The method acquires electromagnetic field data through near-field scanning and reconstructs an equivalent dipole array on the surface of the radiation source by using the source reconstruction method (SRM). In the reconstruction process, the SVD regularization term enhances the algorithm's stability and noise resistance, while the FISTA accelerates the convergence speed.</sec><sec>In order to validate the effectiveness of the proposed method, dipole array reconstruction is first performed using near-field data at a height of 5 mm for a patch antenna simulation model, followed by analyzing the magnetic field data at a 10 mm validation plane. At the 35th iteration, the total relative error of the reconstruction is 1.21%. The influence of the regularization parameter <i>α</i> on the result is then investigated, and it is found that when <i>α</i> = 0.05 the error is minimized. The method is also tested under different Gaussian white noise conditions, and the relative error is kept below 5%, which demonstrates strong robustness.</sec><sec>Finally, the experiments on chips are conducted to verify the method. The proposed method converges stably within 35 iterations, with a relative error of 2.3% in the reconstruction results. The proposed method reduces the total iteration time to 61.7% of the single-layer phaseless interpolation algorithm, while achieving a 52% lower relative error than the double-layer phasless iteration algorithm. The experimental results show that the proposed method can reconstruct phaseless radiation source efficiently and accurately, and has good noise robustness, which is suitable for EMI analysis in ICs.</sec>

Compressed sensing is a technique to recover a sparse vector from its underdetermined linear measurements. Since a naive <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{0}$</tex> optimization approach is hard to tackle due to the discreteness and the non-convexity of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{0}$</tex> norm, a relaxed problem of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}-\ell_{2}$</tex> optimization is often employed for the reconstruction of the sparse vector especially when the measurement noise is not negligible. FISTA (fast iterative shrinkage-thresholding algorithm) is one of popular algorithms for the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}-\ell_{2}$</tex> optimization, and is known to achieve optimal convergence rate among the first order methods. Recently, the employment of optical circuits for various signal processing including deep neural networks has been considered intensively, but it is difficult to implement FISTA with the optical circuit, because it requires operations of divisions with a dynamic value in the algorithm. In this paper, assuming the implementation with the optical circuit, we propose an ADMM (alternating direction method of multipliers) based algorithm for the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}-\ell_{2}$</tex> optimization. It is true that an ADMM based algorithm for the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}-\ell_{2}$</tex> optimization has been already proposed in the literature, but the proposed algorithm is derived with the different formulation from the existing method, and unlike the existing ADMM based algorithm, the proposed algorithm does not include the calculation of the inverse of a matrix. Computer simulation results demonstrate that the proposed algorithm can achieve comparable performance as FISTA or existing ADMM based algorithm while requiring no division operations and no matrix inversions.

Statistical iterative reconstruction in Cone-beam computed tomography (CBCT) uses prior knowledge to form different kinds of regularization terms. The total variation (TV) regularization has shown state-of-the-art performance in suppressing noises and preserving edges. However, it produces the well-known staircase effect. In this paper, a method that involves second-order differential operators was employed to avoid the staircase effect. The ability to avoid staircase effect lies in that higher-order derivatives can avoid over-sharpening the regions of smooth intensity transitions. Meanwhile, a fast iterative shrinkage-thresholding algorithm was used for the corresponding optimization problem. The proposed Hessian Schatten norm-based regularization keeps lots of favorable properties of TV, such as translation and scale invariant, with getting rid of the staircase effect that appears in TV-based reconstructions. The experiments demonstrated the outstanding ability of the proposed algorithm over TV method especially in suppressing the staircase effect.

The construction of brain functional network based on resting-state functional magnetic resonance imaging (fMRI) is an effective method to reveal the mechanism of human brain operation, but the common brain functional network generally contains a lot of noise, which leads to wrong analysis results. In this paper, the least absolute shrinkage and selection operator (LASSO) model in compressed sensing is used to reconstruct the brain functional network. This model uses the sparsity of L1-norm penalty term to avoid over fitting problem. Then, it is solved by the fast iterative shrinkage-thresholding algorithm (FISTA), which updates the variables through a shrinkage threshold operation in each iteration to converge to the global optimal solution. The experimental results show that compared with other methods, this method can improve the accuracy of noise reduction and reconstruction of brain functional network to more than 98%, effectively suppress the noise, and help to better explore the function of human brain in noisy environment.

Image reconstruction for Electrical Resistance Tomography (ERT) is an ill-posed nonlinear inverse problem. Considering the influence of the sparse measurement data on the quality of the reconstructed image, the l1-regularized least-squares program (l1 regularized LSP) is introduced to solve the inverse problem in this paper. To meet the need of high speed in ERT, the fast iterative shrinkage-thresholding algorithm (FISTA) is employed for image reconstruction in our work. Simulation results of the FISTA and l1_ls algorithm show that the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularized LSP is superior to the l2 regularization method, especially in avoiding the over-smoothing of the reconstructed image. In addition, to improve the convergence speed and imaging quality in FISTA algorithm, the initial guess is calculated with the conjugate gradient method. Comparative simulation results demonstrate the feasibility of FISTA in ERT system and its advantage over the l1_ls regularization method.

The reconstruction of cone-beam x-ray luminescence computed tomography (CB-XLCT) is an ill-posed inversion problem because of incomplete data and lack of prior information. To improve the illness of the inversion problem, the data fidelity and regularization term are two key aspects for the reconstruction model. However, there is not much research considering the statistical characteristics of data in XLCT reconstruction, although many various regularizations are studied. To make full use of the data noise model, a strategy combing the maximum likelihood expectation estimation (MLEM) algorithm and the regularization-type algorithm is proposed. In the MLEM algorithm, the Poisson noise is considered for accurate data model. The result by the regularization-type algorithm is used as the specific initial image for the MLEM to improve the reconstruction quality and convergence speed of the MLEM. There are two main steps in the proposed strategy. Firstly, the fast iterative shrinkage-thresholding algorithm (FISTA) with a large regularization parameter is used to get the sparse solution quickly. Secondly, the sparse solution is used as the initial iteration value of the MLEM. The proposed algorithm is named as FISTA-MLEM. Through the stepwise strategy, the image sparsity is guaranteed and the accuracy of the reconstruction is maintained. Result of phantom experiment shows the FISTA-MLEM method presents better contrast to noise ratio and shape similarity compared with other traditional methods, such as ART, Tikhonov, FISTA and TSVD.

The fast iterative shrinkage-thresholding algorithm (FISTA) is a well-known first order method used to minimize <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}$</tex> regularized problems. However, it is also a non-monotone algorithm that can exhibit a sudden and gradual oscillatory behavior during the convergence. One of the parameters that directly affects the convergence of the FISTA method, whose optimal value is typically unknown, is the step-size (SS) that is linked to the Lipschitz constant. Depending on a suitable selection of the SS either manual or automatic, and the SS evolution throughout iterations, e.g. constant, decreasing, or increasing sequence, the practical performance can differ in orders of magnitude with or without stability issues (oscillations or, in the worst case, divergence). In this paper, we propose an algorithm, which has two variants, to address the stability issues in case of ill-chosen parameters for a given SS policy (either manual or adaptive). The proposed method structurally consists of an instability prediction rule based on the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{\infty}$</tex> norm of the gradient, and a correction for it, which can interpreted as an under-relaxation technique.

Spaceborne airborne bistatic linear array synthetic aperture radar (SA-BiLASAR) is a novel type of bistatic SAR (BiSAR) with multi-view three-dimensional (3-D) imaging capabilities. However, due to the separation of transceiver platforms and the structure of linear array antenna, the form of its motion errors is variable. Hence the estimation and compensation of its motion errors are more difficult than traditional SAR system. In this paper, an autofocus method is proposed for SA-BiLASAR 3-D sparse imaging based on maximum sharpness estimation (MSE). Firstly, the sparse imaging model of SA-BiLASAR and the spatial-variant characteristic of its motion errors are discussed. Secondly, combined with the MSE principle, the reconstruct procedure of SA-BiLASAR is presented via fast iterative shrinkage thresholding algorithm (FISTA). Lastly, the complex motion errors of SA-BiLASAR are estimated and compensated by the proposed method. Simulation results demonstrate the effectiveness of the sparse autofocus approach in the case of the different types of motion errors.

Inverse patch transfer functions (iPTF) method is an efficient technique for sound field reconstruction and separation of arbitrarily shaped sound sources in noisy acoustic environments. By applying Neumann boundary condition to a closed virtual cavity surrounding the source, the Helmholtz integral equation is simplified and can be solved with a Green's function satisfying the Neumann boundary condition. However, in the identification of sparsely distributed sources, ghost sources appear in the result solved by using the iPTF method with classic Tikhonov regularization methods and influence the accuracy of identification. In the present work, an evanescent Green's function with fast convergence of calculation is utilized, and a technique that combines the iPTF method and Fast Iterative Shrinkage-Thresholding Algorithm aimed at improving the performance for the identification of sparsely distributed source is proposed. Then double layer measurements, instead of using expensive p-u probes, are employed to acquire the normal velocity of the hologram surface. In numerical simulations, the normal velocities of two anti-phased piston sources are well reconstructed with high sparsity while a disturbing source is radiating in the sound field within the frequency band of 50 to 1000 Hz. Finally, an experiment on two baffled loudspeakers has been carried out. The results of simulations and experiments indicate that the proposed technique has obviously improved the accuracy of the iPTF method in the identification of sparsely distributed sources for the frequency band from 50 to 1000 Hz.