Adaptive energy threshold for monaural speech separation

This paper introduces a speech separation system as a front-end processing step for automatic speech recognition (ASR). It employs computational auditory scene analysis (CASA) to separate the target speech from the interference speech. Specifically, the mixed speech is preprocessed based on auditory peripheral model. Then a pitch tracking is conducted and the dominant pitch is used as a main cue to find the target speech. Next, the time frequency (TF) units are merged into many segments. These segments are then combined into streams via CASA initial grouping. A regrouping strategy is employed to refine these streams via amplitude modulate (AM) cues, which are finally organized by the speaker recognition techniques into corresponding speakers. Finally, the output streams are reconstructed to compensate the missing data in the abovementioned processing steps by a cluster based feature reconstruction. Experimental results of ASR show that at low TMR (<-6dB) the proposed method offers significantly higher recognition accuracy. Index term: speech separation, CASA, speaker recognition, pitch tracking, units grouping, reconstruction

Monaural speech separation is the process of separating the target speech from a noisy speech mixture recorded using single microphone. It can be used in wide range of applications including mobile telephony, hearing aid design and robust automatic speech and speaker recognition (ASR). Recently, researchers use computational auditory scene analysis (CASA) technique to successfully separate the target speech from the monaural noisy speech mixture. In CASA based monaural speech separation techniques, Ideal binary mask (IBM) and Ideal ratio mask (IRM) has been proposed as a computational goal to improve the speech intelligibility and speech quality. This paper reviews and reports various research works carried out using CASA techniques with IBM and IRM to improve speech intelligibility and quality. The experimental results show that CASA systems using IBM improves the speech intelligibility and using IRM improves the speech quality.

The sound of a busy environment, such as a city street, gives rise to a perception of numerous distinct events in a human listener--the 'auditory scene analysis' of the acoustic information. Recent advances in the understanding of this process from experimental psychoacoustics have led to several efforts to build a computer model capable of the same function. This work is known as 'computational auditory scene analysis'. The dominant approach to this problem has been as a sequence of modules, the output of one forming the input to the next. Sound is converted to its spectrum, cues are picked out, and representations of the cues are grouped into an abstract description of the initial input. This 'data-driven' approach has some specific weaknesses in comparison to the auditory system: it will interpret a given sound in the same way regardless of its context, and it cannot 'infer' the presence of a sound for which direct evidence is hidden by other components. The 'prediction-driven' approach is presented as an alternative, in which analysis is a process of reconciliation between the observed acoustic features and the predictions of an internal model of the sound-producing entities in the environment. In this way, predicted sound events will form part of the scene interpretation as long as they are consistent with the input sound, regardless of whether direct evidence is found. A blackboard-based implementation of this approach is described which analyzes dense, ambient sound examples into a vocabulary of noise clouds, transient clicks, and a correlogram-based representation of wide-band periodic energy called the weft. The system is assessed through experiments that firstly investigate subjects' perception of distinct events in ambient sound examples, and secondly collect quality judgments for sound events resynthesized by the system. Although rated as far from perfect, there was good agreement between the events detected by the model and by the listeners. In addition, the experimental procedure does not depend on special aspects of the algorithm (other than the generation of resyntheses), and is applicable to the assessment and comparison of other models of human auditory organization. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

Auditory Scene Analysis: Computational Models

Cocktail-Party Effect with Computational Auditory Scene Analysis — Preliminary Report —

Monaural speech segregation from complex concurrent noise is an extremely challenging problem; binary mask is a method to solve this problem, however, the performance of binary mask is limited by remaining the noise in the result. In this paper, an algorithm integrated Spectral Subtraction and binary masking for speech separation and enhancement was proposed. It follows the framework of computational auditory scene analysis (CASA). The energy of time-frequency (T-F) unit was used as the clue to generate the binary mask; then the spectral subtraction algorithm was used to eliminate noise energy in original speech and an interim speech was obtained, after covered the binary mask on the interim speech, the target speech can be achieved. Systematic evaluation shows that the combined algorithm can stably improve the SNR and voice quality for noisy speech. It performs better than existing binary masking systems in most situations, especially when the noise and the speech have the similar power spectrum.

The segregation of concurrent speakers and other sound sources is an important aspect in improving the performance of audio technology, such as noise reduction and automatic speech recognition, ASR, in difficult acoustic conditions. This technology is relevant for applications like hearing aids, mobile audio devices, robotics, hands-free audio communication and speech-based computer interfaces. Computational auditory-scene analysis (CASA) techniques simulate aspects of processing properties of the human perceptual system using statistical signal-processing techniques to improve inferences about the causes of audio input received by the system. This study argues that CASA is a promising approach to achieve source separation and outlines several theoretical arguments to support this hypothesis. With a focus on computational binaural scene analysis, principles of CASA techniques are reviewed. Furthermore, in an experimental approach, the applicability of a recent model of binaural interaction to improve ASR performance in multi-speaker conditions with spatially separated moving speakers is explored. The binaural model provides input to a statistical inference filter that employs a priori information on possible movements of the sources in order to track the positions of the speakers. The tracks are used to adapt a beamformer that selects a specific speaker. The output of the beamformer is subsequently used for an ASR task. Compared to the unprocessed, that is, mixed, data in a two-speaker condition, the word recognition rates obtained with the enhanced signals based on binaural information were increased from 30.8 to 88.4 %, demonstrating the potential of the proposed CASA-based approach.

In this paper, we propose a new time-frequency mask method for computational auditory scene analysis (CASA) based on convex optimization of the binary mask. In the proposed method, the pitch estimation and segment segregation in conventional CASA are completely replaced by the convex optimization of speech power. Considering the cross-correlation between the power spectra of noisy speech and noise in each of a Gammatone filterbank channel, the objective function of speech power used for convex optimization is built. The speech power is estimated by gradient descent method. Thus, the time-frequency units dominated by speech and noise are labeled by comparing the powers of noisy and estimated speech, and noise. The erroneous local masks are also removed by using the Teager energy of the estimated speech and time-frequency unit smoothing. The results from the average segmental signal-to-noise ratio improvement, HIT-False Alarm rate and subjective test show that the performance of the proposed method outperforms the reference methods.

An approach to segregate the target speech form the mixture utterance in low signal noise ratio (SNR) was proposed. Within the framework of computational auditory scene analysis (CASA), phase was the cue for segregation, and short time Fourier transforms (STFT) was used to extract the phase of the signal. Binary masking was used to group the target speech units based on the difference of phase between the mixture, clean speech and noise. The threshold of the binary masks was not linear. It adapted with the frequency change, and obtained from pretest. Experiments illustrated that the improvement of signal to noise ratio was more than 20dB in babble, m109, white and machinegun noise in -30dB to -20dB. The waveform of the result signal shown it remained most detail of the original signal, and had a well intelligibility. Phase is a robust cue in monaural speech segregation.Keywordsspeech segregationsignal phasecomputational auditory scene analysis (CASA)short time Fourier transforms (STFT)

Speech separation, which has been a challenging task for decades, especially at low signal-to-noise ratios (SNRs), can be cast as a classification problem. In such adverse acoustic environment, extracting robust features from noisy mixtures is crucial for successful classification. In the past studies, features representing temporal dynamics, known as delta features, have been widely used. Combining basic features with their deltas yields better speech separation results than using basic features alone. In this study, the commonly used delta feature was modified according to the characteristics of auditory perception, which included auditory processing on spectral change and spectral contrast. Therefore, we proposed a feature which integrated spectrotemporal context via replacing the commonly used delta feature by spectral change feature and spectral contrast feature. Experimental results showed that the proposed feature could produce better speech segregation performance than the common delta feature.

Computational auditory scene analysis: listening to several things at once

The ideal binary mask (IBM) has been one of the most successful techniques in computational auditory scene analysis (CASA) algorithms. The binary value 1 is assigned to the mask if the local signal-to-noise ratio (SNR) of a particular time-frequency (T-F) units exceeds the local criterion (LC), otherwise the value 0 is assigned to the mask. This binary weighting may discard some parts of the speech during synthesis, which leads to an unnatural sound called musical noise. This paper proposes the optimum soft mask (OSM) to reduce the musical noise, by replacing the hard limiting weights (i.e., 1 or 0) with the variable weights between 0 and 1. The Signal-to-Noise ratio is used as a performance measures to compare the performance of the proposed soft mask with IBM in the context of monaural speech separation. The experimental results show the superior performance of the proposed soft mask as compared with IBM.

A denoising method using computational auditory scene analysis (CASA) for a wearable stethoscope is proposed in this paper. The denoising method imitates human ears to analyze the ambient auditory scene so that it can separate the heart sound signal from unwanted ambient noise. Simulation results show that the method greatly improves the signal noise ratio of the heart sound signal after denoising.

It is relatively easy for a human listener to attend to a particular speaker at a cocktail party in the presence of other speakers, music and environmental sounds. To perform this task, the human listener needs to separate the target speech from a mixture of multiple concurrent sources reflected by various surfaces. This process is referred to as auditory scene analysis. While humans excel at this task using only two ears, machine separation based on two-microphone recordings has proven to be extremely challenging. By incorporating the mechanisms underlying the perception of sound by human listeners, computational auditory scene analysis (CASA) offers a new approach to sound segregation. Binaural hearing – hearing with two ears – employs the difference in sound source locations to improve sound segregation. In this chapter, we describe the principles of binaural processing and review the state-of-the-art in binaural CASA, particularly for speech segregation.

Cocktail party problem is a classical scientific problem that has been studied for decades. Humans have remarkable skills in segregating target speech from a complex auditory mixture obtained in a cocktail party environment. Computational modeling for such a mechanism is however extremely challenging. This chapter presents an overview of several recent techniques for the source separation issues associated with this problem, including independent component analysis/blind source separation, computational auditory scene analysis, model-based approaches, non-negative matrix factorization and sparse coding. As an example, a multistage approach for source separation is included. The application areas of cocktail party processing are explored. Potential future research directions are also discussed.