Threshold reduction for improving Sparse Coding Shrinkage performance in speech enhancement

Sparse coding is a method for finding a neural network representation of multidimensional data in which each of the components of the representation is rarely ignorantly active at the same time. The representation is closely related to independent component analysis (ICA). In this paper, we introduced the basic principle of ICA and have investigated the capabilities of sparse coding shrinkage in the field of image denoising. We have also performed practical implementation of sparse code shrinkage (SCS) and applied to the image denoising. We have seen that SCS outperforms basic denoising methods such as wiener filtering, median filtering and independent component analysis (ICA) applied to image denoising.

Most algorithms for speech enhancement in the spectral domain focus on the acquisition of an estimator of the clean speech parameter, such as spectrum or amplitude. Enhanced speech quality with these methods mainly depends upon the accuracy of the estimator. When the signal-to-noise ratio (SNR) becomes lower, e.g., less than 3 dB, the enhanced speech often shows unsatisfactory quality. Here, we propose a new method for speech enhancement in the spectral domain. It is based on our view that speech characteristics are perceived mainly by part of the spectral components which have higher local (or instantaneous) SNRs. These components have a special importance on speech SNR enhancement and speech understanding. Identifying these components in the noisy spectrum with a decision system and constructing enhanced speech just by the same noisy amplitude and phase, the result is similar to those popular algorithms. All that is needed is a local SNR decision for each spectral component. The threshold for the decision is derived and two popular methods for local SNR (LSNR) estimation are suggested. It shows better performance under lower SNR signals. Though the number of these kind of spectral components varies with different signal SNR, any algorithms which can pick up more than 90% of it is sufficient to match any existing methods. Primary considerations and results are shown.

Generative adversarial networks (GANs) have been increasingly used as feature mapping functions in speech enhancement, in which the noisy speech features are transformed to the clean ones through the generators. This article proposes a novel speech enhancement model based on a cycle-consistent relativistic GAN with Dilated Residual Networks and a Multi-attention mechanism. Using the adversarial loss, improved cycle-consistency losses, and an identity-mapping loss, a noisy-to-clean generator G and an inverse clean-to-noisy generator F simultaneously learn the forward and backward mappings between the source and target domains. To guarantee the stability of the training process, we replace vanilla GAN loss with relativistic average GAN loss and use spectral normalization in discriminators so that they conform to Lipschitz continuity. Furthermore, we employ two attention-based components as multi-attention mechanism to reduce importing signal distortion: attention U-net gates and dilated residual self-attention blocks. By employing these components, our proposed generators can capture long-term inner dependencies between elements of speech features and further preserve linguistic information. Experimental results on a public dataset indicate that the proposed model achieves state-of-the-art speech enhancement performance, especially in reducing speech distortion and improving signal overall quality. Compared with the representative GAN-based approaches, the proposed method significantly achieves the best performance in terms of STOI, CSIG, COVL, and CBAK objective metrics. Moreover, we demonstrate the contribution of each proposed component including relativistic average loss, attention U-net gate, self-attention layers, spectral normalization, and dilation operation by ten comparison systems.

Early fault information of rolling bearings is weak and often submerged by background noise, easily leading to misdiagnosis or missed diagnosis. In order to solve this issue, the present paper puts forward a fault diagnosis method on the basis of adaptive frequency window (AFW) and sparse coding shrinkage (SCS). The proposed method is based on the idea of determining the resonance frequency band, extracting the narrowband signal, and envelope demodulating the extracted signal. Firstly, the paper introduces frequency window, which can slip on the frequency axis and extract the frequency band. Secondly, the double time domain feature entropy is proposed to evaluate the strength of periodic components in signal. The location of the optimal frequency window covering the resonance band caused by bearing fault is determined adaptively by this entropy index and the shifting/expanding frequency window. Thirdly, the signal corresponding to the optimal frequency window is reconstructed, and it is further filtered by the sparse coding shrinkage algorithm to highlight the impact feature and reduce the residue noise. Fourthly, the de-noised signal is demodulated by envelope operation, and the corresponding envelope spectrum is calculated. Finally, the bearing failure type can be judged by comparing the frequency corresponding to the spectral lines with larger amplitude in the envelope spectrum and the fault characteristic frequency. Two bearing vibration signals are applied to validate the proposed method. The analysis results illustrate that this method can extract more failure information and highlight the early failure feature. The data files of Case Western Reserve University for different operation conditions are used, and the proposed approach achieves a diagnostic success rate of 83.3%, superior to that of the AFW method, SCS method, and Fast Kurtogram method. The method presented in this paper can be used as a supplement to the early fault diagnosis method of rolling bearings.

Recently, deep neural networks (DNNs) have become the mainstream strategy for speech enhancement task because it can achieve the higher speech quality and intelligibility than the traditional methods. However, these DNN-based methods always need a large number of parallel corpus consisting of clean speech and noise to produce noisy data for the training of the DNN in order to improve the generalization of the network. As a result, this implies that many noisy speech signals that are collected in real environment cannot be used to train the DNN because of the lack of corresponding clean speech and noise. Additionally, as we know, noise varies with the time and scenario, so we cannot obtain parallel speech and noise due to infinite noise data and some limited speech data. Thus, the network training with unparallel speech and noise data is essential for the generalization of the network. To address this problem, we propose a novel parallel-data-free speech enhancement method, in which the cycle-consistent generative adversarial network (CycleGAN) and multi-objective learning are employed. Our method is also able to make best use of the benefits of multi-objective learning. On the training stage, we utilize two different encoders to encode the features of clean speech and noisy speech, respectively. Then, two forward generators are immediately used to predict the ideal time-frequency (T-F) mask and log-power spectrum (LPS) of clean speech. Two inverse generators are applied to map the magnitude spectrum (MS) and LPS of noisy speech, respectively. In addition, four discriminators are used to distinguish the real speech features from the generated features. Two encoders, four generators and four discriminators are simultaneously trained by using adversarial, identity-mapping, latent similarity and cycle-consistent loss. On the test stage, we directly utilize the forward generators and encoders to acquire the enhanced speech. The experimental results indicate that the proposed approach is able to achieve the better speech enhancement performance than the reference methods. Moreover, the proposed method is also effective to improve speech quality and intelligibility when the networks are trained under the parallel data.

Dynamic controllable speech enhancement models based on quantile loss functions

In this paper we make further foray on the filtering problem for hidden Markov chain, as described in (2). Previous result in the literature on this problem has been obtained in [1], which has derived an optimal nonlinear filter for this problem. The main contribution of this paper is to devise an optimal linear filter for the Markov chain in conjunction with an associated stationary linear filter which amounts here to obtain the convergence of the error covariance matrix. The optimal linear filter derived here bears the following advantages when compared with the one derived in [1]: (i) The innovation coefficients do not depend upon the estimates, which provides a desirable feature of the Kalman filter; (ii) It allows us to derive a stationary filter which has the advantage that it is easy to implement since this filter gain can be performed offline, leading to a linear time-invariant filter. In addition, relying on Euler-Murayama's stochastic numerical method and the results in [2], we carry out a simulation which shows that the filter performs very well.

In this paper, I propose a novel binaural speech enhancement algorithm, called an asynchronous component cancellation Wiener filter (ACC-WF). It cancels not only uncorrelated noise components but also asynchronous noise components, by simply introducing a weight function in the calculation process of a binaural Wiener filter. This weight function is calculated from the phase of the cross-spectrum of observed signals at the left/right ears. In this method, a speech source at the front of a head is regarded as a synchronous signal source, whereas noise sources at different azimuth angles of the speech source are regarded as asynchronous signal sources. In accordance with the weight function, only the asynchronous components are removed. In experiments using speech signals, ACC-WF worked effectively for speech enhancement. The simplicity of the procedure and the effectiveness of ACC-WF may make current binaural speech enhancement systems faster and more reliable.

An efficient speech enhancement algorithm based on spectral classification and Wiener filtering is proposed. Under the assumption that the location of the target source is known, the spectral classification in the frequency domain is performed by the time delay and the magnitude ratio between dual microphone inputs. Then, the Wiener filter is applied to enhance noisy speech. Experimental results show that the proposed algorithm improves the speech quality and the recognition accuracy significantly.

Previously, we presented a speech enhancement algorithm based on the auditorily motivated MPO model. It was shown that the MPO‐based speech enhancement scheme is robust to fluctuating noise. Compared to the output of several other speech enhancement techniques, the MPO‐enhanced speech signals exhibit the lowest increase in Itakura‐Saito, log‐area‐ratio, and log‐likelihood‐ratio‐based distortion measures as the SNR reduces. In the present work, the performance of the MPO speech enhancement is compared with that of the other techniques using subjective evaluations. Three normal‐hearing subjects were recruited to perform two‐alternative, forced‐choice preference tasks. Each sentence was presented twice: one of them was enhanced using the MPO scheme and the other was either left unprocessed or enhanced using one of three other speech enhancement techniques. Sentences are 5‐digits long and corrupted by additive subway noise at −5, 0, 5, or 15‐dB SNR. Most of the time, the MPO‐enhanced speech signal was preferred weakly. Moreover, the MPO‐processed clean speech signal was preferred over clean unprocessed speech signal just as many times as the clean unprocessed speech signal was preferred, indicating that the MPO processing does not introduce any perceptual artifacts in clean speech. [Work supported by NSF BCS0236707 DOD H9823005C0425.]

Deep Neural Network (DNN)-based mask estimation approach is an emerging algorithm in monaural speech enhancement. It is used to enhance speech signals from the noisy background by calculating either speech or noise dominant in a particular frame of the noisy speech signal. It can construct complex models for nonlinear processing. However, the limitation of the DNN-based mask algorithm is a generalization of the targeted population. Past research works focused on their target dataset because of time consumption for the audio recording session. Thus, in this work, different recording conditions were used to study the performance of the DNN-based mask estimation approach. The findings revealed that different language test dataset, as well as different conditions, may not give large impact in speech enhancement performance since the algorithm only learn the noise information. But, the performance of speech enhancement is promising when the trained model has been designed properly, especially given the less sample variations in the input dataset involved during the training session.

The family of state space-based speech enhancement algorithms is taken as the central building block for this thesis; such algorithms include for example Kalman Filters running on autoregressive models of speech. The goal of this thesis is to render this family of algorithms ready, that is, upgrade the existing solutions to make them more flexible and more robust to real-world situations where different types of nonstationary, non-Gaussian noise are to be expected. The chosen context is that of sensitive applications such as hearing aids for which naturalness, intelligibility, and noise reduction are equally important. Most of the state space-based speech enhancement algorithms are currently unable to operate robustly in nonstationary colored-noise environments, and those found in the literature usually assume AWGN conditions. Since in addition, they are in general more computationally demanding than other well-established algorithms, it is not surprising that they are often either overlooked or omitted in respected and recent speech enhancement publications, as the introduction in this thesis will underline. Nevertheless, for AWGN conditions they have still been reported to yield high quality and natural-sounding results, making them an appealing choice for the range of applications chosen. To achieve the goal of this thesis, it is first necessary to categorize, implement and test many state-space based algorithms for speech enhancement; as it turns out, most of the existing algorithms revolve around the same principles and ideas. Then, in this work several tools and extensions of state spaced-based algorithms are developed in order to tackle the required handling of real-world noise. In parallel, since one of the main goals is to preserve naturalness and intelligibility, a variety of configurations based on these proposed tools and extensions are concocted and thoroughly tested so as to be able to determine the best solution both in terms of output quality and complexity requirements. As an additional constraint, most of the tools devised are meant to be compatible with any state space-based algorithm, rendering the work in this thesis homogeneous and unifying. To deal with real-world noise, novel fullband and subband solutions that take into account information from existing noise estimation algorithms are proposed. In the fullband domain, new all-pole modelling techniques are devised, and in the subband domain several alternatives to handle the noise are proposed as well. Throughout the thesis, several important accessory contributions are also developed, such as a new category of particle filtering algorithms for nonlinear speech models, low-cost post-processing techniques for further noise reduction, binaural extensions of monaural state-space models, and various improvement ideas specifically targeted to state-space algorithms running on autoregressive speech models. As an example, a possible state-space based algorithm is proposed based on some of the conclusions of the several Chapters of this thesis, which is shown to outperform three well-established state-of-the-art algorithms in adverse conditions. Rather than merely justifying the work in this thesis by a final arm-wrestling contest, these latter results prove that state-space based algorithms deserve a place amongst highly regarded and viable algorithms for speech enhancement.

Wiener filtering has many applications in the area of imaging science. In image processing, for instance, it is a common way of reducing Gaussian noise. In color science it is often used to estimate reflectances from camera response data on a pixel by pixel basis. Based on a priori assumptions the Wiener filter is the optimal linear filter in the sense of the minimal mean square error to the actual data. In this paper we propose a spatially adaptive Wiener filter to estimate reflectances from images captured by a multispectral camera. The filter estimates pixel noise using local spatial neighborhood and uses this knowledge to estimate a spectral reflectance. In the hypothetical case of a noiseless system, the spatially adaptive Wiener filter equals the standard Wiener filter for reflectance estimation. We present results of various simulation experiments conducted on a multispectral image database using a 6-channel acquisition system and different noise levels.

In this work, a two-dimensional (2-D) representation of the hourly solar radiation data is proposed. The model enables accurate forecasting using image prediction methods. One year solar radiation data that is acquired and collected between August 1, 2005 and July 30, 2006 in Iki Eylul campus of Anadolu University, and a 2-D representation is formed to construct an image data. The data is in raster scan form, so the rows and columns of the image matrix indicate days and hours, respectively. To test the forecasting efficiency of the model, first 1-D and 2-D optimal 3-tap linear filters are calculated and applied. Then, the forecasting is tested through three input one output feed-forward neural networks (NN). One year data is used for training, and 2 month(from August 1,2006 to September 30,2006) for testing. Optimal linear filters and NN models are compared in the sense of root mean square error (RMSE). It is observed that the 2-D model has advantages over the 1- D representation. Furthermore, the NN model accurately converges to forecasting errors smaller than the linear prediction filter results.

Many studies on deep learning-based speech enhancement (SE) utilizing the computational auditory scene analysis method typically employs the ideal binary mask or the ideal ratio mask to reconstruct the enhanced speech signal. However, many SE applications in real scenarios demand a desirable balance between denoising capability and computational cost. In this study, first, an improvement over the ideal ratio mask to attain more superior SE performance is proposed through introducing an efficient adaptive correlation-based factor for adjusting the ratio mask. The proposed method exploits the correlation coefficients among the noisy speech, noise and clean speech to effectively re-distribute the power ratio of the speech and noise during the ratio mask construction phase. Second, to make the supervised SE system more computationally-efficient, quantization techniques are considered to reduce the number of bits needed to represent floating numbers, leading to a more compact SE model. The proposed quantized correlation mask is utilized in conjunction with a 4-layer deep neural network (DNN-QCM) comprising dropout regulation, pre-training and noise-aware training to derive a robust and high-order mapping in enhancement, and to improve generalization capability in unseen conditions. Results show that the quantized correlation mask outperforms the conventional ratio mask representation and the other SE algorithms used for comparison. When compared to a DNN with ideal ratio mask as its learning targets, the DNN-QCM provided an improvement of approximately 6.5% in the short-time objective intelligibility score and 11.0% in the perceptual evaluation of speech quality score. The introduction of the quantization method can reduce the neural network weights to a 5-bit representation from a 32-bit, while effectively suppressing stationary and non-stationary noise. Timing analyses also show that with the techniques incorporated in the proposed DNN-QCM system to increase its compactness, the training and inference time can be reduced by 15.7% and 10.5%, respectively.