Robust zero‐point attraction least mean square algorithm on near sparse system identification

An l 0 -norm constraint Lorentzian (L 0 -CL) algorithm is proposed for adaptive sparse system identification to combat impulsive noise. The L 0 -CL algorithm is derived via exerting an l 0 -norm penalty on the coefficients in the cost function, which is equivalent to add a zero-attractor in the iterations. The zero-attractor attracts the coefficients to zero during the iterations. By the way, the L 0 -CL algorithm can achieve lower mean square error (MSE) for estimating the sparse systems. The simulation results presented in this paper demonstrate that the proposed algorithm has superior performance in both convergence rate and steady-state behavior by identifying the sparse systems in the impulsive noise environment.

We detail in this paper an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm Linearly constrained normalized least-mean-square (L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -CNLMS) algorithm and its weighted version (L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -WCNLMS) applied to solve problems whose solutions have some degree of sparsity, such as the beamforming problem in uniform linear arrays, standard hexagonal arrays, and (non-standard) hexagonal antenna arrays. In addition to the linear constraints present in the CNLMS algorithm, the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -WCNLMS and the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -CNLMS algorithms take into account an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm penalty on the filter coefficients, which results in sparse solutions producing thinned arrays. The effectiveness of both algorithms is demonstrated via computer simulations. When employing these algorithms to antenna array problems, the resulting effect due to the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm constraint is perceived as a large aperture array with few active elements. Although this work focuses the algorithm on antenna array synthesis, its application is not limited to them, i.e., the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -CNLMS is suitable to solve other problems like sparse system identification and signal reconstruction, where the weighted version, the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -WCNLMS algorithm, presents superior performance compared to the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -CNLMS algorithm.

The proportionate normalized least mean square (PNLMS) algorithm is used in sparse system identification for its simplicity and adaptively step-size adjusting scheme. However, the PNLMS has the noisy input and the non-Gaussian output noise problem. A bias Compensated PNLMF algorithm (called BCPNLMF) for identifying sparse system has been proposed to solve aforementioned issues. The BCPNLMF algorithm which takes advantage of the bias compensated and the proportionate scheme, can achieve better steady-state accuracy and faster convergence speed besides identify the system parameters in noisy input and output with non-Gaussian character environments. Simulation results carried out in sparse system identification confirm the remarkable performance of the BCPNLMF, compared with other well-known algorithms.

This paper incorporates a correntropy-induced metric (CIM)-based sparsity constraint into variable step size normalized least mean square (VSSNLMS) algorithm for identification of sparse system corrupted by additive noise. The proposed CIM-VSSNLMS algorithm makes a good trade-off between convergence characteristics, filter stability, and steady state error. The proposed (CIM-VSSNLMS) algorithm incorporates a variable step size that accelerates the convergence characteristics and lowers the normalized misalignment (NMSA) error with respect to CIM-NLMS with constant value of step size. An expression for variant step size is derived under the stability condition of proposed algorithm. Finally, the implementation of the proposed algorithm is carried out in MATLAB software to manifest the improved estimated behavior in the identification of sparse system.

Convergence analysis of sparse LMS algorithms with l1-norm penalty based on white input signal

The reweighted zero-attracting least mean square (RZA-LMS) algorithm has good performance in sparse system identification. However, the convergence performance of the RZA-LMS algorithm degrades in the impulsive noise environment. In this brief, a reweighted zero-attracting least mean M-estimate (RZA-LMM) algorithm is proposed to address the issue. The RZA-LMM algorithm employs the M-estimate cost function with a sparsity-aware penalty term and is derived by using the unconstrained minimization method. In order to further enhance the convergence speed and reduce the steady-state error of the RZA-LMM algorithm, a variable step-size reweighted zero-attracting least mean M-estimate (VSSRZA-LMM) algorithm is proposed. The variable step-size scheme, which adopts the M-estimate cost function with a sparsity-aware penalty term and a constrained term of the step-size, is obtained by using the constrained minimization method. Besides, the mean-square stability is analyzed to obtain the range of the step-size, which guarantees the stabilization of the proposed algorithms. The computer simulation results show that the proposed RZA-LMM algorithm achieves better convergence performance than the RZA-LMS algorithm, and the proposed VSSRZA-LMM algorithm exhibits better convergence performance in terms of convergence speed and steady-state normalized mean square deviation (NMSD) than the RZA-LMM algorithm for sparse system identification in the impulsive noise environment.

A general zero attraction (GZA) proportionate normalized maximum correntropy criterion (GZA-PNMCC) algorithm is devised and presented on the basis of the proportionate-type adaptive filter techniques and zero attracting theory to highly improve the sparse system estimation behavior of the classical MCC algorithm within the framework of the sparse system identifications. The newly-developed GZA-PNMCC algorithm is carried out by introducing a parameter adjusting function into the cost function of the typical proportionate normalized maximum correntropy criterion (PNMCC) to create a zero attraction term. The developed optimization framework unifies the derivation of the zero attraction-based PNMCC algorithms. The developed GZA-PNMCC algorithm further exploits the impulsive response sparsity in comparison with the proportionate-type-based NMCC algorithm due to the GZA zero attraction. The superior performance of the GZA-PNMCC algorithm for estimating a sparse system in a non-Gaussian noise environment is proven by simulations.

Relative rates of solute and pressure propagation into heterogeneous alluvial aquifers following river flow events

A reweighted l 1 -norm penalized set-membership normalized least mean square (RL1SM-NLMS) algorithm is proposed for sparse system identification in this paper. The proposed RL1SM-NLMS algorithm is realized by incorporating a reweighted l 1 -norm constraint into the cost function of the traditional SM-NLMS algorithm to obtain a desired zero attractor term which is used for promoting the convergence for identifying the sparse systems. RL1SM-NLMS algorithm can speed up the convergence and provide a low estimation error floor by means of the zero attractor which is an additional shrinkage in its update equation. The proposed RL1SM-NLMS algorithm is mathematically given and its estimation performance is discussed by using computer simulation in detail. The simulation results obtained from different sparsity level and a echo channel confirm that the proposed RL1SM-NLMS algorithm is effective and robust for sparse system identification in comparison with the existing NLMS, SM-NLMS, ZA-NLMS, ZASM-NLMS and RZASM-NLMS algorithms.

A reweighted l0-norm-constraint LMS algorithm for sparse system identification

A sparse system identification algorithm for network echo cancellation is presented. This new approach exploits both the fast convergence of the improved proportionate normalized least mean square (IPNLMS) algorithm and the efficient implementation of the multidelay adaptive filtering (MDF) algorithm inheriting the beneficial properties of both. The proposed IPMDF algorithm is evaluated using impulse responses with various degrees of sparseness. Simulation results are also presented for both speech and white Gaussian noise input sequences. It has been shown that the IPMDF algorithm outperforms the MDF and IPNLMS algorithms for both sparse and dispersive echo path impulse responses. Computational complexity of the proposed algorithm is also discussed.

A comparison of the energy operator and the Hilbert transform approach to signal and speech demodulation

Effort estimation is an important practice in agile software development. The agile community believes that developers&#x2019; estimates get more accurate over time due to the cumulative effect of learning from short and frequent feedback. However, there is no empirical evidence of an improvement in estimation accuracy over time, nor have prior studies examined effort estimation in different development activities, which are associated with substantial costs. This study fills the knowledge gap in the field of software estimation in agile software development by investigating estimations across time and different development activities based on data collected from a large agile project. This study investigated effort estimation in various development activities, including feature development, bug fixing, and refactoring in agile software development. The results indicate that estimation of agile development does not improve over time, as claimed in the literature. Our data also indicate that no difference exists in the magnitude of estimation errors between feature tasks and bug-fixing/refactoring tasks, while bug-fixing and refactoring tasks are overestimated more frequently than feature tasks. This study also contributes to our knowledge about overestimation and underestimation patterns in agile software development.

Long impulse response system identification presents two challenges for standard normalized least mean square (NLMS) filtering method: heavy computational load and slow convergence. When the response is sparse, partial update algorithms can reduce the computational complexity, but most often at the expense of performance. This paper discusses the tap selection rule for partial update NLMS algorithm in the case of white Gaussian input. We consider output mean square error (MSE) minimization based on gradient analysis and propose an algorithm that switches tap selection criterion between the one based on filter coefficient magnitudes and the one based on input magnitudes. We show that for identifying sparse systems, the new algorithm can outperform standard NLMS significantly with a reduced computational load.

In this paper an ell _1-regularized recursive total least squares (RTLS) algorithm is considered for the sparse system identification. Although recursive least squares (RLS) has been successfully applied in sparse system identification, the estimation performance in RLS based algorithms becomes worse, when both input and output are contaminated by noise (the error-in-variables problem). We proposed an algorithm to handle the error-in-variables problem. The proposed ell _1-RTLS algorithm is an RLS like iteration using the ell _1 regularization. The proposed algorithm not only gives excellent performance but also reduces the required complexity through the effective inversion matrix handling. Simulations demonstrate the superiority of the proposed ell _1-regularized RTLS for the sparse system identification setting.