Robust auditory localization using probabilistic inference and coherence-based weighting of interaural cues.

Robot audition by its own ears (microphones) is essential for natural human-robot communication and interface. Since a microphone is embedded in the head of a robot, the head-related transfer function (HRTF) plays an important role in sound source localization and extraction. Usually, from binaural input, the interaural phase difference (IPD) and interaural intensity difference (IID) are calculated, and then the direction is determined by using IPD and IID with HRTF. The problem of HRTF-based sound source localization is that a HRTF should be measured for each robot in an anechoic chamber, because it depends on the shape of robot's head; HRTF should be interpolated to manipulate a moving talker, because it is available only for discrete azimuth and elevation. To cope with these problems of HRTF, we proposed the auditory epipolar geometry as a continuous function of IPD and IID to dispense with HRTF and have developed a real-time multiple-talker tracking system. This auditory epipolar geometry, however, does not give a good approximation to IID of all range and IPD of peripheral areas. In this paper, the scattering theory in physics is employed to take into consideration the diffraction of sounds around robot's head for better approximation of IID and IPD. The resulting system shows that it is efficient for localization and extraction of sound at higher frequency and from side directions.

A novel deep learning (DL) method is proposed for binaural sound source localization with low SNR. Firstly, the binaural sound signals are decomposed into several channels by using Gammatone filter. Secondly, the 4 feature parameters of Head-related Transfer Function, interaural time difference (ITD), interaural coherence (IC), interaural level difference (ILD), and interaural phase difference (IPD) are extracted. Thirdly, ITD and IC go through a Deep Belief Network (DBN) to determine the quadrant of the sound source and reduce the positioning range. Then, ITD, IC, ILD, and IPD go through a Deep Neural Network (DNN) to obtain the azimuthal angle within 90 degrees. Experimental results show that the proposed algorithm can solve the front-back confusion, and obtain a superior performance with lower complexity and higher precision under low SNR conditions.

Binaural fusion is the perception of a single auditory image when signals are presented to the two ears. Decreasing the interaural coherence of the signals often degrades binaural fusion. However, perception of interaural level differences (ILDs) is minimally affected by interaural coherence changes. This study aims to determine if there is an interaction between ILDs and binaural fusion in the presence of reduced interaural coherence. In Experiment 1, vocoded stimuli with varying envelope interaural coherences were presented containing a range of ILDs. Participants indicated the perceived size and lateralization of the auditory image they perceived. Stimuli with reduced interaural coherence were binaurally fused when there was a large ILD, even when participants indicated that the same stimulus was unfused when there was a 0 dB ILD. This suggests that signals with low interaural coherence can become binaurally fused when the stimulus has large ILDs. To determine if participants could choose to attend to only one ear for stimuli that were reported to be fused in the presence of ILDs in Experiment 1 (i.e. if complete binaural fusion occurred), participants indicated the perceived loudness in their right ear for stimuli with varying right-ear intensities in Experiment 2. When the stimulus had a left-biased ILD, participants had a reduced ability to report changes in right ear signal intensity, suggesting that largely complete binaural fusion was fostered by the presence of large ILDs.

This study investigates whether listeners' cue weighting predicts their real-time use of asynchronous acoustic information in spoken word recognition at both group and individual levels. By focusing on the time course of cue integration, we seek to distinguish between two theoretical views: the associated view (cue weighting is linked to cue integration strategy) and the independent view (no such relationship). The current study examines Seoul Korean listeners' (n = 62) weighting of voice onset time (VOT, available earlier in time) and onset fundamental frequency of the following vowel (F0, available later in time) when perceiving Korean stop contrasts (Experiment 1: cue-weighting perception task) and the timing of VOT integration when recognizing Korean words that begin with a stop (Experiment 2: visual-world eye-tracking task). The group-level results reveal that the timing of the early cue (VOT) integration is delayed when the later cue (F0) serves as the primary cue to process the stop contrast, supporting a relationship between cue weighting and the timing of cue integration (the associated view). At the individual level, listeners with greater reliance on F0 than VOT exhibited a further delayed integration of VOT. These findings suggest that the real-time processing of asynchronously occurring acoustic cues for lexical activation is modulated by the weight that listeners assign to those cues, providing evidence for the associated view of cue integration. This study offers insights into the mechanisms of cue integration and spoken word recognition, and they shed light on variability in cue integration strategies among listeners.

Listeners indicated the intracranial position of bands of noise (from 50 to 400 Hz in width) for several combinations of interaural intensive differences (IID), and interaural temporal differences (ITD), and/or interaural phase differences (IPD). All ITD and IPD combinations produced an interaural delay of 1500 microseconds at the center frequency of the noise. The interaural phase spectra were constructed to produce several patterns of putative cross-correlation functions. Potency of IIDs depended greatly on particular combinations of bandwidth, ITD and IPD. For some combinations, changing the IID by only 3 dB resulted in large shifts in laterality (sometimes moving the image from near one ear to near the other). The complex interactions observed make the results incompatible with the traditional notion that IIDs simply act as weights or scalars. Rather, IIDs act in two distinct manners: (1) as independent scalar quantities and (2) by interacting with specific combinations of bandwidth and ITD/IPD, which is believed to reflect an action within the cross correlation surface.

The auditory system uses three cues to decode sound location: interaural time differences (ITDs), interaural level differences (ILDs), and spectral notches (SNs). Initial processing of these cues is done in separate brainstem nuclei, with ITDs in the medial superior olive, ILDs in the lateral superior olive, and SNs in the dorsal cochlear nucleus. This work addresses the nature of the convergence of localization information in the central nucleus of the inferior colliculus (ICC). Ramachandran et al. (1999) argued that ICC neurons of types V, I, and O, respectively, receive their predominant inputs from ITD-, ILD-, and SN-sensitive brainstem nuclei, suggesting that these ICC response types should be differentially sensitive to localization cues. Here, single-unit responses to simultaneous manipulation of pairs of localization cues were recorded, and the mutual information between discharge rate and individual cues was quantified. Although rate responses to cue variation were generally consistent with those expected from the hypothesized anatomical connections, the differences in information were not as large as expected. Type I units provide the most information, especially about SNs in the physiologically useful range. Type I and O units provide information about ILDs, even at low frequencies at which actual ILDs are very small. ITD information is provided by a subset of all low-frequency neurons. Type V neurons provide information mainly about ITDs and the average binaural intensity. These results are the first to quantify the relative representation of cues in terms of information and suggest a variety of degrees of cue integration in the ICC.

Temporal resolution and temporal integration in the auditory system, with their conflicting time constants, are often regarded as paradox phenomena. In this study a model is presented for the simulation of a number of temporal resolution experiments (e.g., gap detection, forward masking) and classical test tone integration data (cf. contributions by Dau et al.). The model consists of two major parts. A preprocessing stage that transforms the acoustic stimulus into an internal representation, simulating the peripheral processing in the auditory system. Processing errors are accounted for by adding internal noise which introduces detection limits. The second stage is an optimum detector to simulate a ‘central’ decision device. The preprocessing stage consists mainly of three subunits: a Gamma tone filterbank to simulate peripheral filtering; a set of adaptive feedback loops that derives an adapted envelope of every peripheral filter output; a modulation filterbank (per peripheral channel) that analyzes stimulus periodicities in these adapted representations. The model is able to simulate gap detection and test tone integration experiments quantitatively without changing its parameters. It can be shown that within this model the modulation filterbank is vital to achieve this performance. [Supported by the DFG (Az. Mu/1201).]

1. Interaural level differences (ILDs), created by the head and pinna, have long been known to be the dominant acoustic cue for azimuthal localization of high-frequency tones. However, psychophysical experiments have demonstrated that human subjects can also lateralize complex high-frequency sounds on the basis of interaural time differences (ITDs) of the signal envelope. The lateral superior olive (LSO) is one of two pairs of binaural nuclei where the primary extraction of binaural cues for sound source location occurs. "IE" cells in LSO are inhibited by stimuli to the contralateral and excited by stimuli to the ipsilateral ear, and their response rate is therefore dependent on ILD. Anatomic specializations in the afferent pathways to the LSO suggest that this circuit also has a function in the detection of timing cues. We hypothesized that, besides ILD sensitivity, the IE property also conveys a sensitivity to ITDs of amplitude-modulated (AM) tones and could provide the physiological substrate for the psychophysical effect mentioned above. 2. In extracellular recordings from binaural LSO cells in barbiturate-anesthetized cats, response rate was a periodic function of ITDs of AM stimuli, i.e., all cells displayed ITD sensitivity. Binaural responses were smaller than responses to stimulation of the ipsilateral ear alone and were minimal when the envelopes in both ears were in-phase or nearly so. There was good correspondence between responses to ITDs and to dynamic interaural phase differences (IPDs), created by a difference in the envelope frequency to the two ears. Qualitatively, the responses were consistent with the outcome of an IE operation on temporally structured inputs. 3. To compare the relative importance of ILD and ITD, responses to combinations of the two cues were obtained. Despite robust ITD sensitivity in all binaural LSO cells encountered, the changes in response rate that would occur in response to naturally occurring ITDs were small in comparison with the changes expected for naturally occurring ILDs. The main limitation on ITD sensitivity was a steep decline in average discharge rate as the modulation frequency exceeded several hundred Hertz. 4. ITD sensitivity was also present to broadband stimuli, again with minimal rates occurring near 0 ITD. The sensitivity depended in a predictable fashion on the passband of filtered noise and was absent to binaurally uncorrelated noise bands. In response to clicks, ILDs interacted with ITD in a complicated fashion involving amplitude and latency effects. 5. Three low-characteristic frequency (CF) LSO cells were encountered that were IE and showed ITD sensitivity to the fine structure of low-frequency stimuli.(ABSTRACT TRUNCATED AT 400 WORDS)

ObjectiveInteraural level difference (ILD) is the difference in sound pressure level (SPL) between the two ears and is one of the key physical cues used by the auditory system in sound localization. Our current understanding of ILD encoding has come primarily from invasive studies of individual structures, which have implicated subcortical structures such as the cochlear nucleus (CN), superior olivary complex (SOC), lateral lemniscus (LL), and inferior colliculus (IC). Noninvasive brain imaging enables studying ILD processing in multiple structures simultaneously.MethodsIn this study, blood oxygenation level-dependent (BOLD) functional magnetic resonance imaging (fMRI) is used for the first time to measure changes in the hemodynamic responses in the adult Sprague-Dawley rat subcortex during binaural stimulation with different ILDs.Results and SignificanceConsistent responses are observed in the CN, SOC, LL, and IC in both hemispheres. Voxel-by-voxel analysis of the change of the response amplitude with ILD indicates statistically significant ILD dependence in dorsal LL, IC, and a region containing parts of the SOC and LL. For all three regions, the larger amplitude response is located in the hemisphere contralateral from the higher SPL stimulus. These findings are supported by region of interest analysis. fMRI shows that ILD dependence occurs in both hemispheres and multiple subcortical levels of the auditory system. This study is the first step towards future studies examining subcortical binaural processing and sound localization in animal models of hearing.

For speech intelligibility in rooms, the temporal integration of speech reflections is typically modeled by separating the room impulse response (RIR) into an early (assumed beneficial for speech intelligibility) and a late part (assumed detrimental). This concept was challenged in this study by employing binaural RIRs with systematically varied interaural phase differences (IPDs) and amplitude of the direct sound and a variable number of reflections delayed by up to 200 ms. Speech recognition thresholds in stationary noise were measured in normal-hearing listeners for 86 conditions. The data showed that direct sound and one or several early speech reflections could be perfectly integrated when they had the same IPD. Early reflections with the same IPD as the noise (but not as the direct sound) could not be perfectly integrated with the direct sound. All conditions in which the dominant speech information was within the early RIR components could be well predicted by a binaural speech intelligibility model using classic early/late separation. In contrast, when amplitude or IPD favored late RIR components, listeners appeared to be capable of focusing on these components rather than on the precedent direct sound. This could not be modeled by an early/late separation window but required a temporal integration window that can be flexibly shifted along the RIR.

Spatial navigation involves integrating multiple spatial cues, yet the role of feedback in this process remains poorly understood. We examined how corrective feedback affects spatial navigation using landmarks and self-motion cues. Participants performed a spatial localization task under four conditions: landmark-only, self-motion-only, combination (both cues available), and conflict (cues incongruent). Behaviorally, feedback in single-cue conditions reduced response bias and decreased the weight assigned to landmarks, without affecting response variability. Feedback in the combination condition reduced response bias in that condition but induced no other changes. Cognitive modeling revealed that feedback in single-cue conditions reduced sensory biases, which, however, did not translate to greater cue integration. This feedback also reduced the weight assigned to landmarks in different-cause judgments, aligning cue weighting with the maximum-likelihood-estimation principle. This adjustment was accompanied by altered subjective evaluation of cue reliability. Collectively, our findings suggest that rather than enhancing sensory precision or cue integration, corrective feedback shapes spatial cue integration through recalibration mechanisms at multiple levels: aligning perceived spatial location with the true target to reduce sensory bias and adjusting the perception of cue reliability to optimize cue weighting. These findings have broader implications for understanding feedback-driven changes in perceptual and cognitive processes.

Several masking experiments have shown that the auditory system is able to use coherent envelope fluctuations of the masker across frequency within one ear as well as differences in interaural disparity between signal and masker to enhance signal detection. The two effects associated with these abilities are comodulation masking release (CMR) and binaural masking level difference (BMLD). The aim of the present study was to investigate the combination of CMR and BMLD. Thresholds for detecting a sinusoidal signal were measured in a flanking-band paradigm at three different signal frequencies. The masker was presented diotically, and various interaural phase differences (IPDs) of the signal were used. The masker components were either multiplied or Gaussian narrowband noises. In addition, a transposed stimulus was used to increase the BMLD at a high signal frequency. For all frequencies and masker conditions, thresholds decreased as the signal IPD increased and were lower when the masker components were comodulated. The data show an addition of the monaural and binaural masking releases in decibels when masker conditions with and without comodulation and the same spectrum were compared.

It has been known that the auditory motion aftereffect (MAE) is spatially specific [see, for example, D. W. Grantham, Acoustica 84, 337–347 (1998), exp. 3]. However, it still is not very clear which cues for sound localization are responsible for this spatial specificity of the MAE, since several possible cues, such as the spectral profile or interaural level difference, covaried in most past studies. In this study, MAE and its spatial specificity were investigated with sound motion created by using only interaural phase difference (IPD) as a first step to identify the responsible cues. We used the probe method with the method of constant stimuli. Either with or without adaptation to a moving tone, subjects were asked to judge the direction of probe tone motion (0.7 ms duration, either to the left or the right). The slope and the position of the psychometric functions were affected by the direction of the adapter’s motion direction, but only when IPD ranges (spatial ranges of motion) of the adapter and probe tones overlapped each other. These results suggest that the change in IPD is at least one of the cues which produce the spatial specificity.

Modern automation demands mobile robots to be robustly localized in complex scenarios. Current localization systems typically use maps that require to be built and interpreted by experienced operators, increasing deployment costs as well as reducing the adaptability of robots to rearrangements in the environment. In contrast, architectural floor plans can be easily understood by non-expert users and typically represent only the non-rearrangeable parts of buildings. In this paper we propose a system for robot localization in architectural CAD drawings. Our method employs a simultaneous localization and mapping approach to online augment the floor plan with a map represented as a pose-graph with LiDAR measurements. Whenever the environment is accurately mapped in the vicinity of the robot, we use the graph to perform relative localization. We thoroughly evaluate our system in challenging real-world scenarios. Experiments demonstrate that our method is able to robustly track the robot pose even when the floor plan shows major discrepancies from the real-world. We show that our system achieves sub-centimeter accuracy and is suitable for real-time application.

Owls are highly efficient predators with a specialized auditory system designed to aid in the localization of prey. One of the most unique anatomical features of the owl auditory system is the evolution of vertically asymmetrical ears in some species, which improves their ability to localize the elevational component of a sound stimulus. In the asymmetrically eared barn owl, interaural time differences (ITD) are used to localize sounds in azimuth, whereas interaural level differences (ILD) are used to localize sounds in elevation. These two features are processed independently in two separate neural pathways that converge in the external nucleus of the inferior colliculus to form an auditory map of space. Here, we present a comparison of the relative volume of 11 auditory nuclei in both the ITD and the ILD pathways of 8 species of symmetrically and asymmetrically eared owls in order to investigate evolutionary changes in the auditory pathways in relation to ear asymmetry. Overall, our results indicate that asymmetrically eared owls have much larger auditory nuclei than owls with symmetrical ears. In asymmetrically eared owls we found that both the ITD and ILD pathways are equally enlarged, and other auditory nuclei, not directly involved in binaural comparisons, are also enlarged. We suggest that the hypertrophy of auditory nuclei in asymmetrically eared owls likely reflects both an improved ability to precisely locate sounds in space and an expansion of the hearing range. Additionally, our results suggest that the hypertrophy of nuclei that compute space may have preceded that of the expansion of the hearing range and evolutionary changes in the size of the auditory system occurred independently of phylogeny.