Universal Discrete-Domain Speech Enhancement

In this paper, we propose a novel speech enhancement (SE) method by exploiting the discrete wavelet transform (DWT). This new method reduces the amount of fast time-varying portion, viz. the DWT-wise detail component, in the spectrogram of speech signals so as to highlight the speech-dominant component and achieves better speech quality. A particularity of this new method is that it is completely unsupervised and requires no prior information about the clean speech and noise in the processed utterance. The presented DWT-based SE method with various scaling factors for the detail part is evaluated with a subset of Aurora-2 database, and the PESQ metric is used to indicate the quality of processed speech signals. The preliminary results show that the processed speech signals reveal a higher PESQ score in comparison with the original counterparts. Furthermore, we show that this method can still enhance the signal by totally discarding the detail part (setting the respective scaling factor to zero), revealing that the spectrogram can be down-sampled and thus compressed without the cost of lowered quality. In addition, we integrate this new method with conventional speech enhancement algorithms, including spectral subtraction, Wiener filtering, and spectral MMSE estimation, and show that the resulting integration behaves better than the respective component method. As a result, this new method is quite effective in improving the speech quality and well additive to the other SE methods.

Speech enhancement is a process which cultivates the quality of speech signal in noisy environment. It refers to removing or reducing the background noise in order to obtain an improved quality of original speech signal. Degradation of speech signal is most common problem in speech communication, so enhancement of the speech plays a vital role in improving the quality of speech signal. A number of methods are used for speech enhancement. Here we are using LPC based FEM in Wiener filter for speech enhancement. This method is then compared with several speech enhancement algorithms to obtain a better result or better speech quality. On comparing this method with other speech enhancement methods we obtain better speech performance. Here we are using NOIZEUS speech database in order to compare different speech enhancement methods. The experimental result shows that our proposed method provides better result and there is no information loss in original speech signal.

Supervised models for speech enhancement are trained using artificially generated mixtures of clean speech and noise signals. However, the synthetic training conditions may not accurately reflect real-world conditions encountered during testing. This discrepancy can result in poor performance when the test domain significantly differs from the synthetic training domain. To tackle this issue, the UDASE task of the 7th CHiME challenge aimed to leverage real-world noisy speech recordings from the test domain for unsupervised domain adaptation of speech enhancement models. Specifically, this test domain corresponds to the CHiME-5 dataset, characterized by real multi-speaker and conversational speech recordings made in noisy and reverberant domestic environments, for which ground-truth clean speech signals are not available. In this paper, we present the objective and subjective evaluations of the systems that were submitted to the CHiME-7 UDASE task, and we provide an analysis of the results. This analysis reveals a limited correlation between subjective ratings and several supervised nonintrusive performance metrics recently proposed for speech enhancement. Conversely, the results suggest that more traditional intrusive objective metrics can be used for in-domain performance evaluation using the reverberant LibriCHiME-5 dataset developed for the challenge. The subjective evaluation indicates that all systems successfully reduced the background noise, but always at the expense of increased distortion. Out of the four speech enhancement methods evaluated subjectively, only one demonstrated an improvement in overall quality compared to the unprocessed noisy speech, highlighting the difficulty of the task. The tools and audio material created for the CHiME-7 UDASE task are shared with the community Key Terms: Speech Enhancement, CHIME-7 Challenge ,Deep Neural Networks, UDASE Task, Audio Restoration, Subjective Evaluation, objective Evaluation

It has been shown in the literature that the perceptual wavelet packet decomposition (PWPD) and the Teager energy operator (TEO) are useful for various speech processing systems and speech enhancement applications, respectively. By the use of the PWPD and the TEO, this paper presents an improved wavelet-based speech enhancement method. The main advantage of the proposed method is that the over thresholding of speech segments which is usually occurred in conventional wavelet-based speech enhancement schemes can be avoided. As a consequence, the enhanced speech quality of the proposed method can be increased substantially from those of conventional approaches. In addition, the proposed method does not require a complicated estimation of the noise level or any knowledge of the SNR. Using speech signals corrupted by additive and real noises, experimental results demonstrate that the speech enhancement method presented in this paper is capable of outperforming conventional noise cancellation schemes.

Speech enhancement (SE) methods mainly focus on recovering clean speech from noisy input. In real-world speech communication, however, noises often exist in not only speaker but also listener environments. Although SE methods can suppress the noise contained in the speaker’s voice, they cannot deal with the noise that is physically present in the listener side. To address such a complicated but common scenario, we investigate a deep learning-based joint framework integrating noise reduction (NR) with listening enhancement (LE), in which the NR module first suppresses noise and the LE module then modifies the denoised speech, i.e., the output of the NR module, to further improve speech intelligibility. The enhanced speech can thus be less noisy and more intelligible for listeners. Experimental results show that our proposed method achieves promising results and significantly outperforms the disjoint processing methods in terms of various speech evaluation metrics.

Most algorithms for speech enhancement in the spectral domain focus on the acquisition of an estimator of the clean speech parameter, such as spectrum or amplitude. Enhanced speech quality with these methods mainly depends upon the accuracy of the estimator. When the signal-to-noise ratio (SNR) becomes lower, e.g., less than 3 dB, the enhanced speech often shows unsatisfactory quality. Here, we propose a new method for speech enhancement in the spectral domain. It is based on our view that speech characteristics are perceived mainly by part of the spectral components which have higher local (or instantaneous) SNRs. These components have a special importance on speech SNR enhancement and speech understanding. Identifying these components in the noisy spectrum with a decision system and constructing enhanced speech just by the same noisy amplitude and phase, the result is similar to those popular algorithms. All that is needed is a local SNR decision for each spectral component. The threshold for the decision is derived and two popular methods for local SNR (LSNR) estimation are suggested. It shows better performance under lower SNR signals. Though the number of these kind of spectral components varies with different signal SNR, any algorithms which can pick up more than 90% of it is sufficient to match any existing methods. Primary considerations and results are shown.

Deep learning-based speech enhancement methods make use of their nonlinearity properties to estimate the speech and noise signals, especially the nonstationary noise. DCCRN, in particular, achieves state-of-the-art performance on speech intelligibility. However, the nonlinear property also causes concern about the robustness of the method. Novel and unexpected noises can be generated if the noisy input speech is beyond the operation condition of the method. In this paper, we propose a hybrid framework called LDCCRN, which integrates a traditional speech enhancement method LogMMSE-EM and DCCRN. The proposed framework leverages the strength of both approaches to improve the robustness in speech enhancement. While the DCCRN continues to remove the nonstationary noise in the speech, the novel noises generated by DCCRN, if any, are effectively suppressed by LogMMSE-EM. As shown in our experimental results, the proposed method achieves better performance over the traditional approaches measured with standard evaluation methods.

This paper presents a new method for speech enhancement based on a dictionary learning method. The proposed approach is based on using coherence measure in dictionary learning. Data required for better fitting to atoms in sparse representation of noise is provided by a noise estimation algorithm that causes noise dictionary to be trained with the same data size as speech signal. To decrease coherence between dictionaries after the training step, a new method is applied to yield incoherent dictionaries. In sparse representation of speech data, the highest energy atoms of noise dictionary are replaced with the lowest energy atoms, under certain conditions. A similar replacement can happen in sparse representation of noise data. Furthermore, in this paper, only one noise dictionary, chosen by a classification method, is used in speech enhancement step, resulting in a faster algorithm. Objective and subjective measures are used for evaluating the simulation results. According to experimental results, the proposed algorithm has been found superior in performance and computation overhead in comparison with the earlier methods in this context. Moreover, this method achieves significantly better results compared with baseline methods such as multi-band and geometric spectral subtraction.

Previously, we presented a speech enhancement algorithm based on the auditorily motivated MPO model. It was shown that the MPO‐based speech enhancement scheme is robust to fluctuating noise. Compared to the output of several other speech enhancement techniques, the MPO‐enhanced speech signals exhibit the lowest increase in Itakura‐Saito, log‐area‐ratio, and log‐likelihood‐ratio‐based distortion measures as the SNR reduces. In the present work, the performance of the MPO speech enhancement is compared with that of the other techniques using subjective evaluations. Three normal‐hearing subjects were recruited to perform two‐alternative, forced‐choice preference tasks. Each sentence was presented twice: one of them was enhanced using the MPO scheme and the other was either left unprocessed or enhanced using one of three other speech enhancement techniques. Sentences are 5‐digits long and corrupted by additive subway noise at −5, 0, 5, or 15‐dB SNR. Most of the time, the MPO‐enhanced speech signal was preferred weakly. Moreover, the MPO‐processed clean speech signal was preferred over clean unprocessed speech signal just as many times as the clean unprocessed speech signal was preferred, indicating that the MPO processing does not introduce any perceptual artifacts in clean speech. [Work supported by NSF BCS0236707 DOD H9823005C0425.]

Deep learning-based speech enhancement (SE) methods predominantly draw upon two architectural frameworks: generative adversarial networks and diffusion models. In the realm of SE, capturing the local and global relations between signal frames is crucial for the success of these methods. These frameworks typically employ a UNet architecture as their foundational backbone, integrating Long Short-Term Memory (LSTM) networks or attention mechanisms within the UNet to effectively model both local and global signal relations. However, the coupled relation modeling way may not fully harness the potential of these relations. In this paper, we propose an innovative Association-based Fusion Speech Enhancement method (AFSE), a decoupled method. AFSE first constructs a graph that encapsulates the association between each time window of the speech signal, and then models the global relations between frames by fusing the features of these time windows in a manner akin to graph neural networks. Furthermore, AFSE leverages a UNet with dilated convolutions to model the local relations, enabling the network to maintain a high-resolution representation while benefiting from a wider receptive field. Experimental results demonstrate that the AFSE method significantly improves performance in speech enhancement tasks, validating the effectiveness and superiority of our approach. The code is available at https://github.com/jie019/AFSE_IJCAI2025.

This paper investigates a multi-channel denoising autoencoder (DAE)-based speech enhancement approach. In recent years, deep neural network (DNN)-based monaural speech enhancement and robust automatic speech recognition (ASR) approaches have attracted much attention due to their high performance. Although multi-channel speech enhancement usually outperforms single channel approaches, there has been little research on the use of multi-channel processing in the context of DAE. In this paper, we explore the use of several multi-channel features as DAE input to confirm whether multi-channel information can improve performance. Experimental results show that certain multi-channel features outperform both a monaural DAE and a conventional time-frequency-mask-based speech enhancement method.

Speech enhancement (SE) is crucial for improving the intelligibility and quality of noisy speech in applications such as speech recognition and assistive technologies. While traditional SE methods often fail in complex noise environments, deep learning-based approaches and multi-modal techniques have shown significant progress. However, existing multi-modal SE methods, such as audio-visual approaches, are limited in scenarios where visual data cannot be captured. EMG-SENet-basic explores Electromyography (EMG) signals as a robust complementary modality for SE, leveraging their resistance to environmental noise and potential for non-invasive acquisition. In addition, we propose an improved multi-modal SE framework named EMG-SENet-improved featuring a frame-level cross-attention mechanism for dynamic feature fusion, an adaptive weighting mechanism to better integrate amplitude information from both modalities, and the Con-TF-Mamba architecture to enhance local feature extraction in time and frequency domains. Furthermore, Temporal Masking, used to simulate frame loss, improves robustness under real-world conditions. Experimental results demonstrate the proposed method’s superior performance in enhancing SE quality and intelligibility, addressing key challenges in practical applications with degraded or incomplete data.

In this paper, a speech enhancement method based on noise compensation performed on short time magnitude spectra is presented following the geometric approach. Here the noise estimate to be subtracted from the noisy speech spectrum is proposed to be determined by exploiting the low frequency regions of noisy speech of the current frame rather than depending only on the initial silence frames. We argue that this approach of noise estimation offers the capability of tracking the time variation of the non-stationary noise. A noise estimation confidence parameter is also introduced to prevent the subtraction of the overestimated noise. A procedure is thus formulated to compensate for the distortion in the magnitude spectrum of the noisy speech signal. The noise compensated magnitude spectrum is then recombined with the noisy unchanged phase spectrum to produce a modified complex spectrum prior to synthesize an enhanced frame. Extensive simulations are carried out using NOIZEUS database in order to evaluate the performance of proposed method. It is shown in terms of objective measures, spectrogram analysis and informal subjective listening test that the proposed method consistently outperforms one of the state-of-the-art methods of speech enhancement from noisy speech corrupted by babble or car noise at high as well as very low levels of SNR.

Speech communicated is adversely affected by environmental noise. It is important to process the speech and reduce noise for better understanding. Speech enhancement or noise reduction is useful to provide comfort for human or machine listening. Traditional algorithms provide better noise reduction and better-quality speech. Due to the non-stationary nature of noise and the quasi-stationary nature of speech, the traditional methods are proven inadequate in achieving high-quality speech. Later statistical estimators based on Gaussian, and super-Gaussian Probability Density Function (PDF) assumption further improved the enhancement performance. But still, non-stationary noise nature introduces artifacts in processed signal and results in decreased performance. It is observed that neural network approaches and the factorization approach provide better performance even under non-stationary noises by proper training and large database. Different features result in variations in output performance under unseen noise and speaker conditions. It is important to understand the importance and advantages of traditional methods, statistical estimators, and neural network approaches performances. To select the suitable method for a required application, it is essential to consider the trade-off between quality and distortion. In this work, the importance of speech enhancement methods is discussed. Performance measures used for understanding the speech enhancement like Signal to Noise Ratio (SNR), Segmental SNR, Log-Likelihood Ratio (LLR), Weighted Spectral Slope (WSS), Perceptual Evaluation of Speech Quality (PESQ), Short-Time Objective Intelligibility (STOI), Signal to Distortion Ratio (SDR) and Mean Opinion Score (MOS), are given. Highlights of important results are discussed for analyzing better speech enhancement methods for the required application. In this work, performance is compared using objective and subjective performance measures. Simulation results show superior performance when neural network is employed in statistical estimators.

Speech is easily interfered by external environment in reality, which results in the loss of important features. Deep learning has become a popular speech enhancement method because of its superior potential in solving nonlinear mapping problems for complex features. However, the deficiency of traditional deep learning methods is the weak learning capability of important information from previous time steps and long-term event dependencies between the time-series data. To overcome this problem, we propose a novel speech enhancement method based on the fused features of deep neural networks (DNNs) and gated recurrent unit (GRU). The proposed method uses GRU to reduce the number of parameters of DNNs and acquire the context information of the speech, which improves the enhanced speech quality and intelligibility. Firstly, DNN with multiple hidden layers is used to learn the mapping relationship between the logarithmic power spectrum (LPS) features of noisy speech and clean speech. Secondly, the LPS feature of the deep neural network is fused with the noisy speech as the input of GRU network to compensate the missing context information. Finally, GRU network is performed to learn the mapping relationship between LPS features and log power spectrum features of clean speech spectrum. The proposed model is experimentally compared with traditional speech enhancement models, including DNN, CNN, LSTM and GRU. Experimental results demonstrate that the PESQ, SSNR and STOI of the proposed algorithm are improved by 30.72%, 39.84% and 5.53%, respectively, compared with the noise signal under the condition of matched noise. Under the condition of unmatched noise, the PESQ and STOI of the algorithm are improved by 23.8% and 37.36%, respectively. The advantage of the proposed method is that it uses the key information of features to suppress noise in both matched and unmatched noise cases and the proposed method outperforms other common methods in speech enhancement.