Binaural Scene Analysis with Multidimensional Statistical Filters

The segregation of concurrent speakers and other sound sources is an important aspect of the human auditory system but is missing in most current systems for automatic speech recognition (ASR), resulting in a large gap between human and machine performance. The present study uses a physiologically-motivated model of binaural hearing to estimate the position of moving speakers in a noisy environment by combining methods from Computational Auditory Scene Analysis (CASA) and ASR. The binaural model is paired with a particle filter and a beamformer to enhance spoken sentences that are transcribed by the ASR system. Results based on an evaluation in clean, anechoic two-speaker condition shows the word recognition rates to be increased from 30.8% to 72.6%, demonstrating the potential of the CASA-based approach. In different noisy environments, improvements were also observed for SNRs of 5 dB and above, which was attributed to the average tracking errors that were consistent over a wide range of SNRs.

Auditory Scene Analysis: Computational Models

Monaural speech separation is the process of separating the target speech from a noisy speech mixture recorded using single microphone. It can be used in wide range of applications including mobile telephony, hearing aid design and robust automatic speech and speaker recognition (ASR). Recently, researchers use computational auditory scene analysis (CASA) technique to successfully separate the target speech from the monaural noisy speech mixture. In CASA based monaural speech separation techniques, Ideal binary mask (IBM) and Ideal ratio mask (IRM) has been proposed as a computational goal to improve the speech intelligibility and speech quality. This paper reviews and reports various research works carried out using CASA techniques with IBM and IRM to improve speech intelligibility and quality. The experimental results show that CASA systems using IBM improves the speech intelligibility and using IRM improves the speech quality.

This paper introduces a speech separation system as a front-end processing step for automatic speech recognition (ASR). It employs computational auditory scene analysis (CASA) to separate the target speech from the interference speech. Specifically, the mixed speech is preprocessed based on auditory peripheral model. Then a pitch tracking is conducted and the dominant pitch is used as a main cue to find the target speech. Next, the time frequency (TF) units are merged into many segments. These segments are then combined into streams via CASA initial grouping. A regrouping strategy is employed to refine these streams via amplitude modulate (AM) cues, which are finally organized by the speaker recognition techniques into corresponding speakers. Finally, the output streams are reconstructed to compensate the missing data in the abovementioned processing steps by a cluster based feature reconstruction. Experimental results of ASR show that at low TMR (<-6dB) the proposed method offers significantly higher recognition accuracy. Index term: speech separation, CASA, speaker recognition, pitch tracking, units grouping, reconstruction

Cocktail-Party Effect with Computational Auditory Scene Analysis — Preliminary Report —

The sound of a busy environment, such as a city street, gives rise to a perception of numerous distinct events in a human listener--the 'auditory scene analysis' of the acoustic information. Recent advances in the understanding of this process from experimental psychoacoustics have led to several efforts to build a computer model capable of the same function. This work is known as 'computational auditory scene analysis'. The dominant approach to this problem has been as a sequence of modules, the output of one forming the input to the next. Sound is converted to its spectrum, cues are picked out, and representations of the cues are grouped into an abstract description of the initial input. This 'data-driven' approach has some specific weaknesses in comparison to the auditory system: it will interpret a given sound in the same way regardless of its context, and it cannot 'infer' the presence of a sound for which direct evidence is hidden by other components. The 'prediction-driven' approach is presented as an alternative, in which analysis is a process of reconciliation between the observed acoustic features and the predictions of an internal model of the sound-producing entities in the environment. In this way, predicted sound events will form part of the scene interpretation as long as they are consistent with the input sound, regardless of whether direct evidence is found. A blackboard-based implementation of this approach is described which analyzes dense, ambient sound examples into a vocabulary of noise clouds, transient clicks, and a correlogram-based representation of wide-band periodic energy called the weft. The system is assessed through experiments that firstly investigate subjects' perception of distinct events in ambient sound examples, and secondly collect quality judgments for sound events resynthesized by the system. Although rated as far from perfect, there was good agreement between the events detected by the model and by the listeners. In addition, the experimental procedure does not depend on special aspects of the algorithm (other than the generation of resyntheses), and is applicable to the assessment and comparison of other models of human auditory organization. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

One major problem of hearing-impaired people is the cocktail-party effect, i.e., the reduced ability to understand speech in noisy environments. Normal-hearing listeners use binaural information, i.e., interaural level and phase/time differences, in addition to monaural cues to localize sound sources in the environment. Based on the localization result and on the monaural cues, the listener performs a significant noise reduction by concentrating on one of the sources. Noise reduction algorithms have been proposed that try to imitate the binaural information processing in the hearing system by using two microphones at both ears and a central processor. This setup allows for the use of interaural parameters to estimate the direction of sound sources and to attenuate unwanted sound source directions. However, in noisy environments the binaural parameters become noisy as well, reducing the accuracy of the sound source estimation and therefore reducing the maximum performance of the noise reduction algorithm. In this presentation, a localization algorithm is introduced, which uses a statistical description of the binaural parameters in order to perform a robust estimation of sound source directions in noisy environments. Noise reduction schemes based on this localization algorithm are discussed.

Binary masks are essential elements to be used in monaural speech segregation and hearing aids. The performances of the ideal binary masks in terms of signal to noise ratio were evaluated in this article, and a method to predict it before application was proposed. Within the framework of the computational auditory scene analysis (CASA), Ideal binary mask (IBM) has the optimum performance in time-frequency (T-F) units. It can be used as an object goal in global level, which was confirmed by the experiments on a speech mixture database. Furthermore, energy distribution of the target and interfere signals were used together to estimate the performance of IBM in mixture separation.

In this paper, I make the following claims: (1) Subjective experience is tremendously useful in guiding productive research. (2) Studies of auditory scene analysis (ASA) in adults, newborn infants, and non-human animals (e.g., in goldfish or pigeons) establish the generality of ASA and suggest that it has an innate foundation. (3) ASA theory does not favor one musical style over another. (4) The principles used in the composition of polyphony (slightly modified) apply not only to one particular musical style or culture but to any form of layered music. (5) Neural explanations of ASA do not supersede explanations in terms of capacities; the two are complementary. (6) In computational auditory scene analysis (CASA) – ASA by computer systems – or any adequate theory of ASA, the most difficult challenge will be to discover how the contributions of a very large number of types of acoustical evidence and top-down schemas (acquired knowledge about the sound sources in our environments), can be coordinated without producing conflict that disables the system. (7) Finally I argue that the movement of a listener within the auditory scene provides him/her/it with rich information that should not be ignored by ASA theorists and researchers.

In this paper, we propose a new time-frequency mask method for computational auditory scene analysis (CASA) based on convex optimization of the binary mask. In the proposed method, the pitch estimation and segment segregation in conventional CASA are completely replaced by the convex optimization of speech power. Considering the cross-correlation between the power spectra of noisy speech and noise in each of a Gammatone filterbank channel, the objective function of speech power used for convex optimization is built. The speech power is estimated by gradient descent method. Thus, the time-frequency units dominated by speech and noise are labeled by comparing the powers of noisy and estimated speech, and noise. The erroneous local masks are also removed by using the Teager energy of the estimated speech and time-frequency unit smoothing. The results from the average segmental signal-to-noise ratio improvement, HIT-False Alarm rate and subjective test show that the performance of the proposed method outperforms the reference methods.

Speech separation is a process of segregating the target speech from the noisy mixture. Human auditory system naturally has the capability to separate the speech from background noise; however the machine implementation of the same has been failed. Computational Auditory Scene Analysis (CASA) is a recent approach which models the human auditory system by computational means to separate the target speech from the noisy mixture. Most of the speech separation system based on CASA uses energy as one of the reliable feature to segregate the target speech from the noisy mixture. This research work proposes a new approach using adaptive energy selection threshold to segregate the target speech. The experimental result shows significant improvement in the signal to noise ratio as compared to the conventional energy threshold method.

An approach to segregate the target speech form the mixture utterance in low signal noise ratio (SNR) was proposed. Within the framework of computational auditory scene analysis (CASA), phase was the cue for segregation, and short time Fourier transforms (STFT) was used to extract the phase of the signal. Binary masking was used to group the target speech units based on the difference of phase between the mixture, clean speech and noise. The threshold of the binary masks was not linear. It adapted with the frequency change, and obtained from pretest. Experiments illustrated that the improvement of signal to noise ratio was more than 20dB in babble, m109, white and machinegun noise in -30dB to -20dB. The waveform of the result signal shown it remained most detail of the original signal, and had a well intelligibility. Phase is a robust cue in monaural speech segregation.Keywordsspeech segregationsignal phasecomputational auditory scene analysis (CASA)short time Fourier transforms (STFT)

We are engaged in research on computational auditory scene analysis to attain sophisticated robot (computer) human interaction by recognizing auditory awareness. The objective of our research is the understanding of an arbitrary sound mixture including nonspeech sounds and music as well as voiced speech, obtained by robot's ears (or microphones embedded in the robot). The main issues are sound source localization, separation, and recognition at signal processing levels, and signal-to-symbol transformation at the interface level to symbol processing levels. The latter is critical in developmental communication and we are developing an automatic onomatopoeia recognition system. This paper overviews our activities in robot audition, in particular, active direction-pass filter (ADPF) that separates sounds originating from a specific direction by integrating sound source localization and visual processing. ADPF is implemented on three kinds of robots and demonstrates separating and recognizing three simultaneous speeches with a pair of microphones.

Intervention for a Child with Auditory Neuropathy/Dys-synchrony

It is relatively easy for a human listener to attend to a particular speaker at a cocktail party in the presence of other speakers, music and environmental sounds. To perform this task, the human listener needs to separate the target speech from a mixture of multiple concurrent sources reflected by various surfaces. This process is referred to as auditory scene analysis. While humans excel at this task using only two ears, machine separation based on two-microphone recordings has proven to be extremely challenging. By incorporating the mechanisms underlying the perception of sound by human listeners, computational auditory scene analysis (CASA) offers a new approach to sound segregation. Binaural hearing – hearing with two ears – employs the difference in sound source locations to improve sound segregation. In this chapter, we describe the principles of binaural processing and review the state-of-the-art in binaural CASA, particularly for speech segregation.