Binaural Sound Localization Based on Reverberation Weighting and Generalized Parametric Mapping

A Bayes-rule based hierarchical binaural sound source localization system is proposed. By combining three localization cues: interaural time differences (ITDs), interaural intensity differences (IIDs), and spectral cues, and a hierarchical decision making structure, this system enables a sound source to be located in a 3D space by using only the binaural inputs. Preliminary simulations have shown the effectiveness of this system. It can be used in studying binaural localization mechanism and applications such as in hearing aids and robotics.

Binaural sound source localization based on weighted template matching

A novel deep learning (DL) method is proposed for binaural sound source localization with low SNR. Firstly, the binaural sound signals are decomposed into several channels by using Gammatone filter. Secondly, the 4 feature parameters of Head-related Transfer Function, interaural time difference (ITD), interaural coherence (IC), interaural level difference (ILD), and interaural phase difference (IPD) are extracted. Thirdly, ITD and IC go through a Deep Belief Network (DBN) to determine the quadrant of the sound source and reduce the positioning range. Then, ITD, IC, ILD, and IPD go through a Deep Neural Network (DNN) to obtain the azimuthal angle within 90 degrees. Experimental results show that the proposed algorithm can solve the front-back confusion, and obtain a superior performance with lower complexity and higher precision under low SNR conditions.

Bilateral implant recipients theoretically have access to binaural cues. Research in postlingually deafened adults with cochlear implants (CIs) indicates minimal evidence for true binaural hearing. Congenitally deafened children who experience spatial hearing with bilateral CIs, however, might perceive binaural cues in the CI signal differently. There is limited research examining binaural hearing in children with CIs, and the few published studies are limited by the use of unrealistic speech stimuli and background noise. The purposes of this study were to (1) replicate our previous study of binaural hearing in postlingually deafened adults with AzBio sentences in prelingually deafened children with the pediatric version of the AzBio sentences, and (2) replicate previous studies of binaural hearing in children with CIs using more open-set sentences and more realistic background noise (i.e., multitalker babble). The study was a within-participant, repeated-measures design. The study sample consisted of 14 children with bilateral CIs with at least 25 mo of listening experience. Speech recognition was assessed using sentences presented in multitalker babble at a fixed signal-to-noise ratio. Test conditions included speech at 0° with noise presented at 0° (S0N0), on the side of the first CI (90° or 270°) (S0N1stCI), and on the side of the second CI (S0N2ndCI) as well as speech presented at 0° with noise presented semidiffusely from eight speakers at 45° intervals. Estimates of summation, head shadow, squelch, and spatial release from masking were calculated. Results of test conditions commonly reported in the literature (S0N0, S0N1stCI, S0N2ndCI) are consistent with results from previous research in adults and children with bilateral CIs, showing minimal summation and squelch but typical head shadow and spatial release from masking. However, bilateral benefit over the better CI with speech at 0° was much larger with semidiffuse noise. Congenitally deafened children with CIs have similar availability of binaural hearing cues to postlingually deafened adults with CIs within the same experimental design. It is possible that the use of realistic listening environments, such as semidiffuse background noise as in Experiment II, would reveal greater binaural hearing benefit for bilateral CI recipients. Future research is needed to determine whether (1) availability of binaural cues for children correlates with interaural time and level differences, (2) different listening environments are more sensitive to binaural hearing benefits, and (3) differences exist between pediatric bilateral recipients receiving implants in the same or sequential surgeries.

Many kinds of sound source localization systems have been developed for detecting a direction of sound source. They are commonly using time delay of arrival (TDOA) or interaural time difference (ITD) algorithm for sound source localization where, especially, the ITD is the difference in arrival time of a sound between two ears. It is largely changed depending on frequency components of sound even though the sound source is located in the same place. In this paper we propose a binaural sound localization system using sparse coding based ITD (SITD) and self-organizing map (SOM). The sparse coding is used for decomposing given sounds into three components: time, frequency and magnitude. Moreover we estimate the azimuth angle through the SOM. This localization system is installed in our robot that has two ears, head and body. We use PeopleBot as a body of the robot.

We applied motion theory to robot audition to improve the inadequate performance. Motions are critical for overcoming the ambiguity and sparseness of information obtained by two microphones. To realise this, we first designed a sound source localisation system integrated with cross-power spectrum phase (CSP) analysis and an EM algorithm. The CSP of sound signals obtained with only two microphones was used to localise the sound source without having to measure impulse response data. The expectation-maximisation (EM) algorithm helped the system to cope with several moving sound sources and reduce localisation errors. We then proposed a way of constructing a database for moving sounds to evaluate binaural sound source localisation. We evaluated our sound localisation method using artificial moving sounds and confirmed that it could effectively localise moving sounds slower than 1.125 rad/s. Consequently, we solved the problem of distinguishing whether sounds were coming from the front or rear by rotating and/or tipping the robot's head that was equipped with only two microphones. Our system was applied to a humanoid robot called SIG2, and we confirmed its ability to localise sounds over the entire azimuth range as the success rates for sound localisation in the front and rear areas were 97.6% and 75.6% respectively.

Binaural sound source localization (SSL) determines the position of a sound source based on three types of cues that contain head-related transfer functions. Many audio processing systems in our daily work and life rely on SSL such as speech enhancement/recognition, and human-robot interaction to name a few. However, the accuracy of SSL under adverse acoustic scenarios is still hard to be ensured. This paper proposes long short-term memory with a feature extraction technique based on scattering decomposition to improve the estimation accuracy of the loudspeaker position. The obtained results demonstrate that the proposed method achieves great performance in multiple noisy environments in comparison to recent literature methods considering reduced sequence data. Finally, the proposed method shows the capability of regression output to estimate azimuth and elevation angles using only 27% of localizations for training.

The role of interaural time differences in infants' sound localization was investigated. One experiment on free-field sound localization corroborated previous findings that the minimum audible angle changes substantially toward the end of the first half year after birth, and 3 experiments explored interaural time discrimination in that age range. The first of these 3 experiments used an adaptive psychophysical procedure, showing that infants responded appropriately in a sound lateralization task that provides a direct measure of interaural time discrimination. The other 2 experiments improved on the psychophysical procedure by taking into account the ceiling level on performance in the task. Infants aged 16, 20, and 28 weeks had thresholds in the range of 50 to 75 microsec, with no apparent age difference. These thresholds were much lower than would be predicted from studies of free-field sound localization, indicating that sensitivity to interaural time differences is not a limiting factor for the precision of sound localization in this age range. Instead, age-related changes in free-field sound localization may reflect the need to integrate across different localization cues and to calibrate the changing values of cues due to head growth.

A novel end-to-end binaural sound localisation approach is proposed which estimates the azimuth of a sound source directly from the waveform. Instead of employing hand-crafted features commonly employed for binaural sound localisation, such as the interaural time and level difference, our end-to-end system approach uses a convolutional neural network (CNN) to extract specific features from the waveform that are suitable for localisation. Two systems are proposed which differ in the initial frequency analysis stage. The first system is auditory-inspired and makes use of a gammatone filtering layer, while the second system is fully data-driven and exploits a trainable convolutional layer to perform frequency analysis. In both systems, a set of dedicated convolutional kernels are then employed to search for specific localisation cues, which are coupled with a localisation stage using fully connected layers. Localisation experiments using binaural simulation in both anechoic and reverberant environments show that the proposed systems outperform a state-of-the-art deep neural network system. Furthermore, our investigation of the frequency analysis stage in the second system suggests that the CNN is able to exploit different frequency bands for localisation according to the characteristics of the reverberant environment.

There are many applications that would be aided by the determination of the physical position and orientation of users. Some of the applications include service robots, video conference, intelligent living environments, security systems and speech separation for hands-free communication devices (Coen, 1998; Wax & Kailath, 1983; Mungamuru & Aarabi, 2004; Sasaki et al., 2006; Lv & Zhang 2008). As an example, without the information on the spatial location of users in a given environment, it would not be possible for a service robot to react naturally to the needs of the user. To localize a user, sound source localization techniques are widely used (Nakadai et al., 2000; Brandstein & Ward, 2001; Cheng & Wakefield, 2001; Sasaki et al., 2006). Sound localization is the process of determining the spatial location of a sound source based on multiple observations of the received sound signals. Current sound localization techniques are generally based upon the idea of computing the time difference of arrival (TDOA) information with microphone arrays (Knnapp & Cater, 1976; Brandstein & Silverman, 1997). An efficient method to obtain TDOA information between two signals is to compute the cross-correlation of the two signals. The computed correlation values give the point at which the two signals from separate microphones are at their maximum correlation. When only two isotropic (i.e., not directional as in the mammalian ear) microphones are used, the system experiences front-back confusion effect: the system has difficulty in determining whether the sound is originating from in front of or behind the system. A simple and efficient method to overcome this problem is to incorporate more microphones (Huang et al., 1999). Various weighting functions or pre-filters such as Roth, SCOT, PHAT, Eckart filter and HT can be used to increase the performance of time difference estimation (Knnapp & Cater, 1976). However, the performance improvement is achieved with the penalty of large power consumption and hardware overhead, which may not be suitable for the implementation of portable systems such as service robots. In this chapter, we propose an efficient sound source localization method under the assumption that three isotropic microphones are used to avoid the front-back confusion

In voice-based human-computer interaction, various environmental noise may interfere with the normal transmission of information. Noises may typically lead to performance degradation and efficiency reduction of the voice interaction system. Aiming for reliable sound source localization and effective speech enhancement, we researched those algorithms in scenario of voice-based human-computer interaction with indoor intelligent robots, carried out analysis and implementation. In this paper, we analyzed the localization algorithms based on steered-response power, improved, implemented, and tested on it for its stability in practical application. The results indicated the good performance of the algorithm we proposed under the condition that no strong directional noise interferes with the target signal. Reliable two-dimensional sound localization information can be provided, ensuring real-time performance of the system. We also implemented the speech enhancement algorithm based on fixed beamforming, using the sound localization information provided by the two-dimensional sound localization algorithm. The results showed that the algorithm has a certain inhibitory effect on non-correlative noise on the premise of reliable sound source localization and undistorted signal of speech.

Sound localization in the barn owl relies on the early processing of two binaural cues, ITD (interaural time differences) and ILD (interaural level differences), that takes place in two parallel pathways in the auditory brainstem. Previous modeling studies fell into one of two classes, either they used biophysically plausible neural models on simplified binaural stimuli with artificially induced ITDs, or abstract cross-correlation models on realistic acoustical inputs. Here we present a functional spiking neural model of the ITD processing pathway of the barn owl, with a realistic virtual acoustic environment. The acoustical environment was reproduced by filtering natural sounds through measured Head-Related Transfer Functions (HRTFs) in the barn owl (Tyto Alba). Basilar membrane filtering was modeled with a bank of linear gammachirp filters and compression. Monaural neurons in the Nucleus Magnucellularis (NM) receive inputs from auditory nerve fibers and project axons to binaural neurons in the Nucleus Laminaris (NL), with various conduction delays, so that best delays depend on preferred frequency in accordance with measured distributions [1]. Empirical findings show that the best tuning of NL neurons is invariant to level and ILD changes [3], which implies that the spike timing of their monaural inputs should also be invariant to level. This is a challenging problem, since integration to a fixed threshold inevitably results in earlier spike timing when the input level is increased. We address this issue by using a spiking model with dynamic threshold [2,4]. We derive analytical conditions under which one can achieve this level invariance, and show through numerical simulations that the responses of our model to realistic inputs are indeed robust to level and ILD changes. The azimuth is estimated from the activation pattern of NL neurons. We found that the model could accurately estimate the azimuth of a large set of natural sounds with various levels, and significant background noise. Furthermore, we studied the robustness of ITD estimation in our model to reflective environments. We believe this study is a first step towards the development of neural models of sound localization in ecological environments.

Finding the source of a sound by ear is called sound localization, and most animals, including nocturnal owls, that use sound signals for communication, prey capture, and predator evasion can localize sound. Sound localization and its brain mechanisms are better known in the barn owl than in any other animal. The barn owl can localize sound equally well in two dimensions, azimuth, or the horizontal plane, and elevation, or the vertical plane. Experiments show that the barn owl uses interaural time difference for determining sound azimuth and interaural intensity difference for elevation. When a sound signal reaches one ear before the other ear, an interaural time difference results. Sound travels 1 cm in 29 μsec. The maximum interaural time difference experienced by the barn owl is about 170 μsec when sound propagates along the aural axis; the minimum time difference is zero when sound emanates from a place equidistant to the two ears. Thus, interaural time disparity varies systematically as a function of sound azimuth. Interaural intensity difference is due to the head's shadow in the sound field. The ear in the shadow registers a weaker signal than the other ear.KeywordsInferior ColliculusCochlear NucleusSound LocalizationInteraural Time DifferenceNucleus MagnocellularisThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Behavioral sensitivity to interaural time differences in the rabbit

Compared to all other hearing animals, insects are the smallest ones, both in absolute terms and in relation to the wavelength of most biologically relevant sounds. The ears of insects can be located at almost any possible body part, such as wings, legs, mouthparts, thorax or abdomen. The interaural distances are generally so small that cues for directional hearing such as interaural time and intensity differences (IITs and IIDs) are also incredibly small, so that the small body size should be a strong constraint for directional hearing. Yet, when tested in behavioral essays for the precision of sound source localization, some species demonstrate hyperacuity in directional hearing and can track a sound source deviating from the midline by only [Formula: see text]-[Formula: see text]. They can do so by using internally coupled ears, where sound pressure can act on both sides of a tympanic membrane. Here we describe their varying anatomy and mode of operation for some insect groups, with a special focus on crickets, exhibiting probably one of the most sophisticated of all internally coupled ears in the animal kingdom.