Noise-regularized adaptive filtering for speech enhancement

The complexity of the speech enhancement problem has motivated many different solutions. However, most techniques address situations in which the target speech is fully intelligible and the background noise energy is low in comparison with that of the speech. Thus while current enhancement algorithms can improve the perceived quality, the intelligibility of the speech is not increased significantly and may even be reduced. Recent research shows that intelligibility of very noisy speech can be improved by the use of a binary mask, in which a binary weight is applied to each time-frequency bin of the input spectrogram. There are several alternative goals for the binary mask estimator, based either on the Signalto-Noise Ratio (SNR) of each time-frequency bin or on the speech signal characteristics alone. Our approach to the binary mask estimation problem aims to preserve the important speech cues independently of the noise present by identifying time-frequency regions that contain significant speech energy. The speech power spectrum varies greatly for different types of speech sound. The energy of voiced speech sounds is concentrated in the harmonics of the fundamental frequency while that of unvoiced sounds is, in contrast, distributed across a broad range of frequencies. To identify the presence of speech energy in a noisy speech signal we have therefore developed two detection algorithms. The first is a robust algorithm that identifies voiced speech segments and estimates their fundamental frequency. The second detects the presence of sibilants and estimates their energy distribution. In addition, we have developed a robust algorithm to estimate the active level of the speech. The outputs of these algorithms are combined with other features estimated from the noisy speech to form the input to a classifier which estimates a mask that accurately reflects the time-frequency distribution of speech energy even at low SNR levels. We evaluate a mask-based speech enhancer on a range of speech and noise signals and demonstrate a consistent increase in an objective intelligibility measure with respect to noisy speech.

Robust speech recognition using singular value decomposition based speech enhancement

This paper proposes an unsupervised DNN-based speech enhancement approach founded on deep priors (DPs). Here, DP signifies that DNNs are more inclined to produce clean speech signals than noises. Conventional methods based on DP typically involve training on a noisy speech signal using a random noise feature as input, stopping training only a clean speech signal is generated. However, such conventional approaches encounter challenges in determining the optimal stop timing, experience performance degradation due to environmental background noise, and suffer a trade-off between distortion of the clean speech signal and noise reduction performance. To address these challenges, we utilize two DNNs: one to generate a clean speech signal and the other to generate noise. The combined output of these networks closely approximates the noisy speech signal, with a loss term based on spectral kurtosis utilized to separate the noisy speech signal into a clean speech signal and noise. The key advantage of this method lies in its ability to circumvent trade-offs and early stopping problems, as the signal is decomposed by enough steps. Through evaluation experiments, we demonstrate that the proposed method outperforms conventional methods in the case of white Gaussian and environmental noise while effectively mitigating early stopping problems.

Diffusion models are a new class of generative models that have shown outstanding performance in image generation literature. As a consequence, studies have attempted to apply diffusion models to other tasks, such as speech enhancement. A popular approach in adapting diffusion models to speech enhancement consists in modelling a progressive transformation between the clean and noisy speech signals. However, one popular diffusion model framework previously laid in image generation literature did not account for such a transformation towards the system input, which prevents from relating the existing diffusion-based speech enhancement systems with the aforementioned diffusion model framework. To address this, we extend this framework to account for the progressive transformation between the clean and noisy speech signals. This allows us to apply recent developments from image generation literature, and to systematically investigate design aspects of diffusion models that remain largely unexplored for speech enhancement, such as the neural network preconditioning, the training loss weighting, the stochastic differential equation (SDE), or the amount of stochasticity injected in the reverse process. We show that the performance of previous diffusion-based speech enhancement systems cannot be attributed to the progressive transformation between the clean and noisy speech signals. Moreover, we show that a proper choice of preconditioning, training loss weighting, SDE and sampler allows to outperform a popular diffusion-based speech enhancement system while using fewer sampling steps, thus reducing the computational cost by a factor of four.

Recently, deep neural networks (DNNs) have become the mainstream strategy for speech enhancement task because it can achieve the higher speech quality and intelligibility than the traditional methods. However, these DNN-based methods always need a large number of parallel corpus consisting of clean speech and noise to produce noisy data for the training of the DNN in order to improve the generalization of the network. As a result, this implies that many noisy speech signals that are collected in real environment cannot be used to train the DNN because of the lack of corresponding clean speech and noise. Additionally, as we know, noise varies with the time and scenario, so we cannot obtain parallel speech and noise due to infinite noise data and some limited speech data. Thus, the network training with unparallel speech and noise data is essential for the generalization of the network. To address this problem, we propose a novel parallel-data-free speech enhancement method, in which the cycle-consistent generative adversarial network (CycleGAN) and multi-objective learning are employed. Our method is also able to make best use of the benefits of multi-objective learning. On the training stage, we utilize two different encoders to encode the features of clean speech and noisy speech, respectively. Then, two forward generators are immediately used to predict the ideal time-frequency (T-F) mask and log-power spectrum (LPS) of clean speech. Two inverse generators are applied to map the magnitude spectrum (MS) and LPS of noisy speech, respectively. In addition, four discriminators are used to distinguish the real speech features from the generated features. Two encoders, four generators and four discriminators are simultaneously trained by using adversarial, identity-mapping, latent similarity and cycle-consistent loss. On the test stage, we directly utilize the forward generators and encoders to acquire the enhanced speech. The experimental results indicate that the proposed approach is able to achieve the better speech enhancement performance than the reference methods. Moreover, the proposed method is also effective to improve speech quality and intelligibility when the networks are trained under the parallel data.

Speech can be expressed as a mechanism of expressing thoughts and ideas using vocal sounds. In humans, speech signals are generated at vocal cords, travelled through the vocal tract, and finally produced & transmitted through speaker’s mouth. These speech signals then usually travels through air or other mediums to the listener’s ear, where it acts as pressure waves. The bandwidth of speech signal is around 4 KHz. The noise produced by various ambient sources such as vehicles normally lies in this frequency range. Therefore, speech signals get easily distorted by the ambient noise. These distorted or degraded speech signals are called noisy speech signals. This paper focuses on speech processing (in particularly speech enhancement) of the noisy speech signals. This is very important as speech is the most commonly used way of communication and interaction between humans; however, it is very complex to understand. Therefore, this paper proposes an adaptive algorithm based on Wiener filter for speech enhancement. The proposed adaptive Wiener filter depends on the adaptation of the filter transfer function from sample to sample based on the speech signal statistics (mean and variance). The adaptive Wiener filter is implemented in time domain rather than in frequency domain. This is done to accommodate the random speech signal. The proposed method is compared to the traditional Wiener filter and the spectral subtraction methods.

Speech signals are complex in nature with respect to other forms of communication media such as text or image. Different forms of noises (e.g., additive noise, channel noise, babble noise) interfere with the speech signals and drastically hamper the quality of the speech in the noisy speech signals. Enhancement of speech signals is a daunting task considering multiple forms of noises while denoising speech signals. Certain analog noise eliminator models have been studied over the years for this purpose. Researchers have also delved into some deep learning techniques to enhance speech signals. In this study, a speech enhancement system is investigated using Convolutional Denoising Autoencoder (CDAE). It takes advantages from the 2D structured inputs of the features extracted from speech signals and also considers the local temporal relationship among the features. In the proposed system, CDAE is trained considering features from noisy speech signal as input and clean speech features as desired output. The proposed CDAE based method has been tested on a benchmark dataset, called Speech Command Dataset, and attained 80% similarity between denoised speech and actual clean speech. The proposed system achieved perceptual evaluation of speech quality (PESQ) value of 2.43 which outperformed other related existing methods.

This paper proposes a Variational Mode Decomposition (VMD) based approach for enhancement of speech signals distorted by white Gaussian noise. VMD is a data adaptive method which decomposes the signal into intrinsic mode functions (IMFs) by using the Alternating Direction Method of Multipliers (ADMM). Each IMF or mode will contain a center frequency and its harmonics. This paper tries to explore VMD as a Speech enhancement technique. In the proposed method, the noisy speech signal is decomposed into IMFs using VMD. The noisy IMFs are enhanced using two methods; VMD based wavelet shrinkage (VMD-WS) and VMD based MMSE log STSA (VMD-MMSE). The speech signal distorted with different noise levels are enhanced using the VMD based methods. The level of noise reduction and speech signal quality are measured using the objective quality measures.

A speech enhancement system for noisy speech signals under different environments, namely airport, babble, car, exhibition, restaurant, station, street and train, is designed and implemented in this paper. Spectral subtraction is the most popular method for speech enhancement which suffers from perceptually annoying musical noise. Musical noise is reduced here by using low variance spectrum estimators and enhancement is achieved by wavelet thresholding the multitaper spectrum of the speech. The main feature of the proposed system is a novel switching mechanism to an optimal wavelet filter bank or wavelet packet filter bank matching to the critical bands of the human ear, based on the type of noise present in the input speech. The system is evaluated using noisy speech signals under different environment noises at different signal to noise ratios. The subjective and objective test results show that the proposed method improves the quality and intelligibility of speech signals.

enhancement is a task to improve the intelligibility and perceptual quality of degraded speech signal. Recently, neural networks based methods have been applied to speech enhancement. However, many neural network based methods require noisy and clean speech pairs for training. We propose a speech enhancement framework that can be trained with large-scale weakly labelled AudioSet dataset. Weakly labelled data only contain audio tags of audio clips, but not the onset or offset times of speech. We first apply pretrained audio neural networks (PANNs) to detect anchor segments that contain speech or sound events in audio clips. Then, we randomly mix two detected anchor segments containing speech and sound events as a mixture, and build a conditional source separation network using PANNs predictions as soft conditions for speech enhancement. In inference, we input a noisy speech signal with the one-hot encoding of Speech as a condition to the trained system to predict enhanced speech. Our system achieves a PESQ of 2.28 and an SSNR of 8.75 dB on the VoiceBank-DEMAND dataset, outperforming the previous SEGAN system of 2.16 and 7.73 dB respectively.

We present an adaptive enhancement method for speech signal corrupted by the 3 Tesla (3T) functional Magnetic Resonance Imaging (fMRI) acoustic noise. The noisy speech signal is filtered into several frequency bands and noise cancellation is applied to each band independently. We use Normalized Least Mean Squares (NLMS) algorithm for updating the co-efficients of the adaptive filters in both wideband and in multi-band implementations. The NLMS algorithm is used to ensure faster convergence and stability. The multi-band method shows a better performance than the conventional wideband method resulting in a better noise reduction. It is shown that the distortion introduced by the band-pass filters limits the overall speech enhancement when the number of bands increases beyond an optimal value.

Speech enhancement technology has been applied to many research fields of speech signal processing, such as speech coding, speech recognition, noise suppression and so on. However, in practical applications, there is always noise in the environment, voice in the transmission, to avoid noise interference, and the quality of the environment noise has different effects on the quality of the speech signal. Therefore, speech enhancement algorithm can realize the original clean speech from noisy speech signal. The traditional single microphone in the process of dealing with environmental noise, the received target signal and noise signals are often overlapping in the time and frequency spectrum. In view of the traditional method, using microphone array can improve the quality of voice communication, and can make full use of the time and space information of speech signal. Therefore, the use of a microphone array can overcome the shortcomings of a single microphone.

Noise estimation and suppression is very important for improving the quality of speech signal. Noises exist in almost all places. In reality, more than one noise degrades the speech signal. It is hard to find and supress various types of noise that affect the speech quality. This paper proposed a method for noise estimation of mixed non-stationary noisy speech signal. This method uses Spectral properties of the noisy speech signal to detect the frequency regions of noise signal. Highest frequency of speech signal is calculated and it is considered as the threshold value for separating noise signal and clean speech signal. Using Spectral subtraction, Standard deviation of noise spectrum is subtracted with noisy spectrum to acquire enhanced speech signal. Performance of the method is evaluated using SNR and Spectrogram. The main focus of this paper is to propose an independent method which estimates the noise of any type and nature.

In this paper, we present an implementation of adaptive Wiener filtering as a speech enhancement technique for the pitch detection purpose from speech signals affected by noise. The adaptive Wiener filtering is used as a pre-processing stage for speech enhancement, and then a combined technique from Auto-Correlation Function (ACF) and Average Magnitude Difference Function (AMDF) is implemented to get accurate results. The main objective is to improve the process of detecting the fundamental frequency of the speech signal. The adaptive Wiener filter shows a superiority in the proposed pitch detection method as compared to the traditional Wiener filterandspectral subtraction.

In this paper, a speech enhancement method based on noise compensation performed on short time magnitude spectra is presented following the geometric approach. Here the noise estimate to be subtracted from the noisy speech spectrum is proposed to be determined by exploiting the low frequency regions of noisy speech of the current frame rather than depending only on the initial silence frames. We argue that this approach of noise estimation offers the capability of tracking the time variation of the non-stationary noise. A noise estimation confidence parameter is also introduced to prevent the subtraction of the overestimated noise. A procedure is thus formulated to compensate for the distortion in the magnitude spectrum of the noisy speech signal. The noise compensated magnitude spectrum is then recombined with the noisy unchanged phase spectrum to produce a modified complex spectrum prior to synthesize an enhanced frame. Extensive simulations are carried out using NOIZEUS database in order to evaluate the performance of proposed method. It is shown in terms of objective measures, spectrogram analysis and informal subjective listening test that the proposed method consistently outperforms one of the state-of-the-art methods of speech enhancement from noisy speech corrupted by babble or car noise at high as well as very low levels of SNR.