Speech enhancement using a wavelet thresholding method based on symmetric Kullback–Leibler divergence

This paper introduces a new speech enhancement algorithm based on the adaptive threshold of intrinsic mode functions (IMFs) of noisy signal frames extracted by empirical mode decomposition. Adaptive threshold values are estimated by using the gamma statistical model of Teager energy operated IMFs of noisy speech and estimated noise based on symmetric Kullback–Leibler divergence. The enhanced speech signal is obtained by a semisoft thresholding function, which is utilized by threshold IMF coefficients of noisy speech. The method is tested on the NOIZEUS speech database and the proposed method is compared with wavelet-shrinkage and EMD-shrinkage methods in terms of segmental SNR improvement (SegSNR), weighted spectral slope (WSS), and perceptual evaluation of speech quality (PESQ). Experimental results show that the proposed method provides a higher SegSNR improvement in dB, lower WSS distance, and higher PESQ scores than wavelet-shrinkage and EMD-shrinkage methods. The proposed method shows better performance than traditional threshold-based speech enhancement approaches from high to low SNR levels.

In this paper, a two-stage dual tree complex wavelet packet transform (DTCWPT) based speech enhancement algorithm has been proposed, in which a speech presence probability (SPP) estimator and a generalized minimum mean squared error (MMSE) estimator are developed. To overcome the drawback of signal distortions caused by down sampling of wavelet packet transform (WPT), a two-stage analytic decomposition concatenating undecimated wavelet packet transform (UWPT) and decimated WPT is employed. An SPP estimator in the DTCWPT domain is derived based on a generalized Gamma distribution of speech, and Gaussian noise assumption. The validation results show that the proposed algorithm can obtain enhanced perceptual evaluation of speech quality (PESQ), and segmental signal-to-noise ratio (SegSNR) at low signal-to-noise ratio (SNR) nonstationary noise, compared with four other state-of-the-art speech enhancement algorithms, including optimally modified log-spectral amplitude (OM-LSA), soft masking using a posteriori SNR uncertainty (SMPO), a posteriori SPP based MMSE estimation (MMSE-SPP), and adaptive Bayesian wavelet thresholding (BWT).

Speech communicated is adversely affected by environmental noise. It is important to process the speech and reduce noise for better understanding. Speech enhancement or noise reduction is useful to provide comfort for human or machine listening. Traditional algorithms provide better noise reduction and better-quality speech. Due to the non-stationary nature of noise and the quasi-stationary nature of speech, the traditional methods are proven inadequate in achieving high-quality speech. Later statistical estimators based on Gaussian, and super-Gaussian Probability Density Function (PDF) assumption further improved the enhancement performance. But still, non-stationary noise nature introduces artifacts in processed signal and results in decreased performance. It is observed that neural network approaches and the factorization approach provide better performance even under non-stationary noises by proper training and large database. Different features result in variations in output performance under unseen noise and speaker conditions. It is important to understand the importance and advantages of traditional methods, statistical estimators, and neural network approaches performances. To select the suitable method for a required application, it is essential to consider the trade-off between quality and distortion. In this work, the importance of speech enhancement methods is discussed. Performance measures used for understanding the speech enhancement like Signal to Noise Ratio (SNR), Segmental SNR, Log-Likelihood Ratio (LLR), Weighted Spectral Slope (WSS), Perceptual Evaluation of Speech Quality (PESQ), Short-Time Objective Intelligibility (STOI), Signal to Distortion Ratio (SDR) and Mean Opinion Score (MOS), are given. Highlights of important results are discussed for analyzing better speech enhancement methods for the required application. In this work, performance is compared using objective and subjective performance measures. Simulation results show superior performance when neural network is employed in statistical estimators.

In this chapter, we will detail a new speech enhancement technique based on Lifting Wavelet Transform (LWT) and Artifitial Neural Network (ANN). This technique also uses the MMSE Estimate of Spectral Amplitude. It consists at the first step in applying the LWTto the noisy speech signal in order to obtain two noisy details coefficients, cD1 and cD2 and one approximation coefficient, cA2. After that, cD1 and cD2 are denoised by soft thresholding and for their thresholding, we need to use suitable thresholds, thrj,1≤j≤2. Those thresholds, thrj,1≤j≤2, are determined by using an Artificial Neural Network (ANN). The soft thresholding of those coefficients, cD1 and cD2, is performed in order to obtain two denoised coefficients, cDd1 and cDd2 . Then the denoising technique based on MMSE Estimate of Spectral Amplitude is applied to the noisy approximation cA2 in order to obtain a denoised coefficient, cAd2. Finally, the enhanced speech signal is obtained from the application of the inverse of LWT, LWT−1 to cDd1, cDd2 and cAd2. The performance of the proposed speech enhancement technique is justified by the computations of the Signal to Noise Ratio (SNR), Segmental SNR (SSNR) and Perceptual Evaluation of Speech Quality (PESQ).

Speech enhancement is an important pre-processing task in the area of speech processing research. Many techniques have been applied in this area since four/five decades. With progressive research it occupies a special position in various fields like engineering, medicine, society and security. Adaptive algorithms found effective for such cases and are utilized in this problem. The work is based on decomposition method using variational mode decomposition (VMD) technique, where the decomposed components signify the frequency characteristics of the signal. Since Wiener filtering is used in VMD inherently, it is modified with the least mean squares (LMS) adaptive algorithm for good accuracy and adaptability in this work. Different noises like Babble noise, Street noise, and Exhibition noise are considered and the corresponding signals are decomposed into five intrinsic mode functions (IMFs). Basically, the lower modes are of high frequency and noisy; whereas the higher mode IMFs contain the low and medium frequency components and are considered as the enhanced signal. The results of the proposed algorithm are found excellent as compared to earlier techniques. The resultant wave forms are visually observed and the sound is verified for audible range. Also different measuring parameters are considered for its performance measure. It is measured in terms of signal-to-noise ratio (SNR), segmental signal to noise ratio (SegSNR), perceptual evaluation of speech quality (PESQ) and log spectral distance (LSD). The technique is verified with standard database NOIZEUS for 0, 5, 10, 15 dB respectively and also in real world case.

Masking properties have been widely exploited in speech enhancement techniques, especially those implemented in the spectral domain. The incorporation of auditory masking in a subspace technique invariably requires a transformation linking the frequency and eigendomains. In this letter, an invertible transformation between the frequency and eigendomains is derived. The proposed transformation is verified through a conventional masking-based subspace speech-enhancement method. Simulation results show that our proposed transformation for speech enhancement outperforms the conventional transformation in terms of segmental signal-to-noise ratio (SNR), perceptual evaluation of speech quality (PESQ), and listening tests.

Chapter 2 - Recursive noise estimation-based Wiener filtering for monaural speech enhancement

The speech enhancement effect of traditional deep learning algorithms is not ideal under low signal-to-noise ratios (SNR). Skip connections-deep neural network (Skip-DNN) improves the traditional deep neural network (DNN) by adding skip connections between each layer of the neural network to solve the degradation problem of DNN. In this paper, the Multiresolution Cochleagram (MRCG) features in the gammachirp transform domain are denoised to obtain the improved MRCG (I-MRCG). The noise reduction method adopts the Minimum Mean-Square Error Short-Time Spectral Amplitude Estimator (MMSE-STSA) and takes I-MRCG as the input feature and Skip-DNN as the training network to improve the speech enhancement effect of the model. This paper also proposes an improved source-to-distortion ratio (SDR) loss function. When the loss function uses the improved SDR, it will improve the performance of Skip-DNN speech enhancement model. The experiments in this paper are performed on the Edinburgh dataset. When using I-MRCG as the input feature of Skip-DNN, the average perceptual evaluation of speech quality (PESQ) is 2.9137, and the average short-time objective intelligibility (STOI) is 0.8515. Compared with MRCG as Skip-DNN input features, the improvements are 0.91% and 0.71%, respectively. When the improved SDR is used as the loss function of the speech model, the average PESQ is 2.9699 and the average STOI is 0.8547. Compared with other loss functions, the improved SDR has a better enhancement effect when used as the loss function of the speech enhancement model.

In information processing using Wavelet transform, wavelet subband coefficients are often modelled by a probability distribution function. Recently, a local energy histogram method has been proposed to alleviate the difficulty in modeling wavelet subband coefficients with a previously assumed distribution function. Actually, the similarity between any two local energy histograms was measured by a symmetrized Kullback-Leibler divergence (SKLD). However, this measurement neglects the balance of wavelet subbands’ roles in texture classification. In this paper, we propose an efficient texture classification method based on weighted symmetrized Kullback-Leibler divergences (WSKLDs) between two local energy histograms (LEHs). In particular, for any test and training images, we index their Wavelet subbands in the same way, and weight the SKLD between any two LEHs of the s-th wavelet subbands of two image by the reciprocal of the summation of the SKLDs between the expected LEHs of any two different texture classes over all training images. Experimental results reveal that our proposed method outperforms five state-of-the-art methods.KeywordsTexture classificationWavelet subbandImbalance problemLocal energy histogram (LEH)Kullback-Leibler divergence (KLD)

The paper investigates the hidden relationships among speech samples by applying graph tools. Specifically, we first estimate an applicable graph topology for unstructured speech signals, which can map speech signals into the vertex domain successfully and construct as Speech graph signals (SGSs). On the basis, we define a new graph Fourier transform for SGSs, which can investigate its related graph Fourier analysis. Moreover, we propose a new Graph structure spectral subtraction (GSSS) method for speech enhancement under different noisy environments. Simulation results show that the performance of the GSSS method can be significantly improved than the classical Basic spectral subtraction (BSS) method in terms of the average Segmental signal-to-noise ratio (SSNR), Perceptual evaluation of speech quality (PESQ) and the computational complexity.

This study proposes a bi-directional recurrent neural network (Bi-RNN) post-processing method for speech enhancement (SE) at low signal-to noise ratios (SNR). Current speech enhancement solutions performed badly under low SNR situations. Loizou and Kim proposed a solution to reduce speech distortion errors in time-frequency (T-F) domain but it requires the knowledge of ground truth. As ground truth is unknown in real-life applications, the current study proposes to use a Bi-RNN to implement Loizou and Kim’s solution as a post-processing method for SE engines. Our solutions do not require prior knowledge of ground truth. The effectiveness of the proposed method is investigated with a spectral subtraction (SS) SE engine, a non-negative matrix factorization (NMF) SE engine, and a deep neural network ideal ratio mask (DNN-IRM) SE engine, under matched/mis-matched noise and different SNR conditions. Experimental results demonstrate that the proposed post-processing method effectively improved both perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) for all of these SE engines, especially at low SNR conditions.

Purpose: The purpose of this study was to investigate and analyze to increase the quality and intelligibility of speech in cars. The passenger dialogue inside the car, the sound of other equipment, and a wide range of interference effects are major challenges in the task of speech separation in-car environment.Methods: Speech enhancement based on the source separation algorithm has been proposed to enhance the preferred speech signals using a microphone array inside a car. The proposed approach determines the signal direction in the time domain by utilizing the time difference of arrival (TDOA). TDOA signals are processed, and an adaptive least mean square method is used to determine the enhanced preferred signal.Results: Experimental results show that the proposed approach yields an signal-to-noise ratio (SNR) of 7.4, perceptual evaluation of speech quality (PESQ) of 2.33 respectively. The proposed strategy outperforms existing methods in terms of PESQ, and SNR. The PESQ of the proposed method is 2.45%, and 5.22% better than the existing Independent Component Analyses, and Run length Source techniques.Conclusion: Finally, compared to the existing methods, the suggested speech enhancement algorithm is more reliable and flexible, and it can properly identify the exact location of the sound.

Recent work in the domain of speech enhancement has explored the use of self-supervised speech representations to aid in the training of neural speech enhancement models. However, much of this work focuses on using the deepest or final outputs of self supervised speech representation models, rather than the earlier feature encodings. The use of self supervised representations in such a way is often not fully motivated. In this work it is shown that the distance between the feature encodings of clean and noisy speech correlate strongly with psychoacoustically motivated measures of speech quality and intelligibility, as well as with human Mean Opinion Score (MOS) ratings. Experiments using this distance as a loss function are performed and improved performance over the use of STFT spectrogram distance based loss as well as other common loss functions from speech enhancement literature is demonstrated using objective measures such as perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI).

Deep Neural Networks (DNNs) have been successfully adopted as a regression model in speech enhancement. However, the performance in the battlefield environment is not always satisfactory because the noise energy is often dominating in certain speech segments causing speech distortion. For the speech enhancement in complicated battlefield environment where multiple noises can simultaneously corrupt speech, such as gunshots and explosions, we propose an enhanced method to improve the existing DNN-based speech enhancement by using Recurrent Neural Networks (RNNs). This RNN model judges whether each frame is in a low SNR state, and then fuses two DNN-based speech enhancement models. The proposed method is compared with existing DNN-based speech enhancement techniques through the perceptual evaluation of speech quality (PESQ) and the short-time objective intelligibility (STOI) scores in various noisy speech conditions. The experimental results demonstrate significant improvements over the state-of-the-art techniques and reflect the usability of the method in a real battlefield environment.KeywordsSpeech enhancementDeep Neural Networks (DNNs)Recurrent Neural Networks (RNNs)Battlefield environment

In this paper, we considered the problem of the speech enhancement similar to the real-world environments where several complex noise sources simultaneously degrade the quality and intelligibility of a target speech. The existing literature on the speech enhancement principally focuses on the presence of one noise source in mixture signals. However, in real-world situations, we generally face and attempt to improve the quality and intelligibility of speech where various complex stationary and nonstationary noise sources are simultaneously mixed with the target speech. Here, we have used deep learning for speech enhancement in complex-noisy environments and used ideal binary mask (IBM) as a binary classification function by using deep neural networks (DNNs). IBM is used as a target function during training and the trained DNNs are used to estimate IBM during enhancement stage. The estimated target function is then applied to the complex-noisy mixtures to obtain the target speech. The mean square error (MSE) is used as an objective cost function at various epochs. The experimental results at different input signal-to-noise ratio (SNR) showed that DNN-based complex-noisy speech enhancement outperformed the competing methods in terms of speech quality by using perceptual evaluation of speech quality (PESQ), segmental signal-to-noise ratio (SNRSeg), log-likelihood ratio (LLR), weighted spectral slope (WSS). Moreover, short-time objective intelligibility (STOI) reinforced the better speech intelligibility.