Variance normalized perceptual subspace speech enhancement with noise estimation using SPP

Speech enhancement using fourth-order cumulants and optimum filters in the subband domain

In this paper, we propose a robust speech enhancement algorithm for nonstationary noise environments, which comprises a multiband spectral subtraction (MBSS) speech estimator and a minima controlled recursive averaging (MCRA) noise estimation. A multiband approach is obtained by dividing the whole spectrum into multiple subbands and applying spectral subtraction independently in each band. The noise estimate exploit the observation that the noise signal typically has a nonuniform effect on the spectrum of speech and is given by averaging the past spectrum power value and using a time and frequency dependent smoothing factor that is calculated based on the speech presence probability in subbands. We test the performance of proposed algorithm in various nonstationary noise environments. The experiment results confirm the superiority of the MBSS and MCRA estimator. Outstanding speech enhancement is achieved, while avoiding the residual musical noise phenomena.

In this paper, a speech enhancement method based on noise compensation performed on short time magnitude spectra is presented following the geometric approach. Here the noise estimate to be subtracted from the noisy speech spectrum is proposed to be determined by exploiting the low frequency regions of noisy speech of the current frame rather than depending only on the initial silence frames. We argue that this approach of noise estimation offers the capability of tracking the time variation of the non-stationary noise. A noise estimation confidence parameter is also introduced to prevent the subtraction of the overestimated noise. A procedure is thus formulated to compensate for the distortion in the magnitude spectrum of the noisy speech signal. The noise compensated magnitude spectrum is then recombined with the noisy unchanged phase spectrum to produce a modified complex spectrum prior to synthesize an enhanced frame. Extensive simulations are carried out using NOIZEUS database in order to evaluate the performance of proposed method. It is shown in terms of objective measures, spectrogram analysis and informal subjective listening test that the proposed method consistently outperforms one of the state-of-the-art methods of speech enhancement from noisy speech corrupted by babble or car noise at high as well as very low levels of SNR.

The interchannel phase difference (IPD) may be one of the most widely-used spatial cues in multichannel speech processing, and has been used in beamformers and post filters for speech enhancement. The coherence, which is also used as a feature for speech enhancement, can provide information on the reliability of the IPD for the estimation of the speech presence probability (SPP). In this paper, we propose dual microphone speech enhancement adopting <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a posteriori</i> SPP estimation based on statistical modeling of the IPD. The marginal distribution of the IPD is derived from the distribution of the relative transfer function which is parameterized with the IPD and coherence, with a single assumption that the observed discrete Fourier transform (DFT) coefficients in each frequency are distributed according to a complex bivariate Gaussian distribution. Given the direction of arrival of the desired signal, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a posteriori</i> SPP is obtained using the IPD distributions with and without the information on the location of the interfering source, and is applied to speech enhancement. Experimental results for various types and locations of noise, signal-to-noise ratios, reverberation times, and locations of the target source showed that the proposed method outperformed previously proposed approaches utilizing IPD information.

Hybrid probabilistic adaptation mode controller for generalized sidelobe cancellers applied to multi-microphone speech enhancement

Speech enhancement for non-stationary noise environments

Speech enhancement (SE) is crucial for improving the intelligibility and quality of noisy speech in applications such as speech recognition and assistive technologies. While traditional SE methods often fail in complex noise environments, deep learning-based approaches and multi-modal techniques have shown significant progress. However, existing multi-modal SE methods, such as audio-visual approaches, are limited in scenarios where visual data cannot be captured. EMG-SENet-basic explores Electromyography (EMG) signals as a robust complementary modality for SE, leveraging their resistance to environmental noise and potential for non-invasive acquisition. In addition, we propose an improved multi-modal SE framework named EMG-SENet-improved featuring a frame-level cross-attention mechanism for dynamic feature fusion, an adaptive weighting mechanism to better integrate amplitude information from both modalities, and the Con-TF-Mamba architecture to enhance local feature extraction in time and frequency domains. Furthermore, Temporal Masking, used to simulate frame loss, improves robustness under real-world conditions. Experimental results demonstrate the proposed method’s superior performance in enhancing SE quality and intelligibility, addressing key challenges in practical applications with degraded or incomplete data.

Recently, deep neural networks (DNNs) have become the mainstream strategy for speech enhancement task because it can achieve the higher speech quality and intelligibility than the traditional methods. However, these DNN-based methods always need a large number of parallel corpus consisting of clean speech and noise to produce noisy data for the training of the DNN in order to improve the generalization of the network. As a result, this implies that many noisy speech signals that are collected in real environment cannot be used to train the DNN because of the lack of corresponding clean speech and noise. Additionally, as we know, noise varies with the time and scenario, so we cannot obtain parallel speech and noise due to infinite noise data and some limited speech data. Thus, the network training with unparallel speech and noise data is essential for the generalization of the network. To address this problem, we propose a novel parallel-data-free speech enhancement method, in which the cycle-consistent generative adversarial network (CycleGAN) and multi-objective learning are employed. Our method is also able to make best use of the benefits of multi-objective learning. On the training stage, we utilize two different encoders to encode the features of clean speech and noisy speech, respectively. Then, two forward generators are immediately used to predict the ideal time-frequency (T-F) mask and log-power spectrum (LPS) of clean speech. Two inverse generators are applied to map the magnitude spectrum (MS) and LPS of noisy speech, respectively. In addition, four discriminators are used to distinguish the real speech features from the generated features. Two encoders, four generators and four discriminators are simultaneously trained by using adversarial, identity-mapping, latent similarity and cycle-consistent loss. On the test stage, we directly utilize the forward generators and encoders to acquire the enhanced speech. The experimental results indicate that the proposed approach is able to achieve the better speech enhancement performance than the reference methods. Moreover, the proposed method is also effective to improve speech quality and intelligibility when the networks are trained under the parallel data.

In this study, a novel human acoustic perception motivated Wiener filter speech enhancement system is presented to cope with real-world interfering background noises. Guiding by the speech presence probability, two alternative methods are proposed to reduce the noise by adopting the audible sound pressure level (SPL) and the masking characteristic of the human auditory system to achieve better listening comfort level. More specifically, when the probability of speech presence in the noisy signal is less than the decision threshold, a new SPL compressed method effectively reduces the noise. When the speech presence probability is more than the decision threshold, an improved acoustical mask threshold constrained Wiener filter approach enhances the noisy speech. Moreover, in order to evaluate the performance of the new system, the proposed algorithm is compared with the classic prior signal-to-noise ratio-based Wiener filter and three acoustic perception related algorithms. The experimental results show that the proposed algorithm significantly outperforms the four comparing algorithms in terms of speech quality and intelligibility either in stationary or moderate non-stationary noisy environments. Thus, the intended approach can be employed as the front-end module for various speech-related applications.

A reliable estimator for speech presence probability (SPP) can significantly improve the performance of many speech enhancement algorithms. Previous work showed that a good SPP estimator can be obtained by using a smooth a-posteriori signal to noise ratio (SNR) function, which can be achieved by reducing the noise variance when estimating the speech power spectrum. In this paper, a wavelet based denoising algorithm is proposed for such purpose. We first apply the wavelet transform to the periodogram of a noisy speech signal to generate an oracle for indicating the locations of the noise floor in the periodogram. We then make use of that oracle to selectively remove the wavelet coefficients of the noise floor in the log multitaper spectrum (MTS) of the noisy speech. The remaining wavelet coefficients are then used to reconstruct a denoised MTS and in turn generate a smooth a-posteriori SNR function. Simulation results show that the new SPP estimator outperforms the traditional approaches and enables a significantly improvement in the quality and intelligibility of the enhanced speeches. © 2012 IEEE.

In this paper, a two-stage dual tree complex wavelet packet transform (DTCWPT) based speech enhancement algorithm has been proposed, in which a speech presence probability (SPP) estimator and a generalized minimum mean squared error (MMSE) estimator are developed. To overcome the drawback of signal distortions caused by down sampling of wavelet packet transform (WPT), a two-stage analytic decomposition concatenating undecimated wavelet packet transform (UWPT) and decimated WPT is employed. An SPP estimator in the DTCWPT domain is derived based on a generalized Gamma distribution of speech, and Gaussian noise assumption. The validation results show that the proposed algorithm can obtain enhanced perceptual evaluation of speech quality (PESQ), and segmental signal-to-noise ratio (SegSNR) at low signal-to-noise ratio (SNR) nonstationary noise, compared with four other state-of-the-art speech enhancement algorithms, including optimally modified log-spectral amplitude (OM-LSA), soft masking using a posteriori SNR uncertainty (SMPO), a posteriori SPP based MMSE estimation (MMSE-SPP), and adaptive Bayesian wavelet thresholding (BWT).

Diffusion models are a new class of generative models that have shown outstanding performance in image generation literature. As a consequence, studies have attempted to apply diffusion models to other tasks, such as speech enhancement. A popular approach in adapting diffusion models to speech enhancement consists in modelling a progressive transformation between the clean and noisy speech signals. However, one popular diffusion model framework previously laid in image generation literature did not account for such a transformation towards the system input, which prevents from relating the existing diffusion-based speech enhancement systems with the aforementioned diffusion model framework. To address this, we extend this framework to account for the progressive transformation between the clean and noisy speech signals. This allows us to apply recent developments from image generation literature, and to systematically investigate design aspects of diffusion models that remain largely unexplored for speech enhancement, such as the neural network preconditioning, the training loss weighting, the stochastic differential equation (SDE), or the amount of stochasticity injected in the reverse process. We show that the performance of previous diffusion-based speech enhancement systems cannot be attributed to the progressive transformation between the clean and noisy speech signals. Moreover, we show that a proper choice of preconditioning, training loss weighting, SDE and sampler allows to outperform a popular diffusion-based speech enhancement system while using fewer sampling steps, thus reducing the computational cost by a factor of four.

In this paper, we propose a novel speech enhancement (SE) method by exploiting the discrete wavelet transform (DWT). This new method reduces the amount of fast time-varying portion, viz. the DWT-wise detail component, in the spectrogram of speech signals so as to highlight the speech-dominant component and achieves better speech quality. A particularity of this new method is that it is completely unsupervised and requires no prior information about the clean speech and noise in the processed utterance. The presented DWT-based SE method with various scaling factors for the detail part is evaluated with a subset of Aurora-2 database, and the PESQ metric is used to indicate the quality of processed speech signals. The preliminary results show that the processed speech signals reveal a higher PESQ score in comparison with the original counterparts. Furthermore, we show that this method can still enhance the signal by totally discarding the detail part (setting the respective scaling factor to zero), revealing that the spectrogram can be down-sampled and thus compressed without the cost of lowered quality. In addition, we integrate this new method with conventional speech enhancement algorithms, including spectral subtraction, Wiener filtering, and spectral MMSE estimation, and show that the resulting integration behaves better than the respective component method. As a result, this new method is quite effective in improving the speech quality and well additive to the other SE methods.

We propose time-frequency domain methods for noise estimation and speech enhancement. A speech presence detection method is used to find connected time-frequency regions of speech presence. These regions are used by a noise estimationmethod and both the speech presence decisions and the noise estimate are used in the speech enhancement method. Different attenuation rules are applied to regions with and without speech presence to achieve enhanced speech with natural sounding attenuated background noise. The proposed speech enhancement method has a computational complexity, which makes it feasible for application in hearing aids. An informal listening test shows that the proposed speech enhancement method has significantly higher mean opinion scores than minimum mean-square error log-spectral amplitude (MMSE-LSA) and decision-directed MMSE-LSA.

We propose an approach to compute the inter-subband correlation (ISBC) of noisy speech signals to distinguish between speech and noise segments in the time-frequency plane. The proposed spectral correlation estimator provides information about the input signal which can be used to derive a binary mask or the speech-presence probability. Unlike other approaches it does not require an estimate of the noise power. To this end we analyse a received noisy speech signal in the modulation domain and identify similarly modulated subband signals within a range of modulation frequencies that are typical for speech signals. Based on this pre-processing step, we identify a single reference subband that most likely contains speech and estimate the spectral correlations with respect to this reference band. The algorithm proposed in this paper aims at a very low computational complexity which makes it suitable for hearing aids.