Localization of multiple sound sources based on a CSP analysis with a microphone array

This paper describes a method for estimating Direction Of Arrival (DOA) of multiple sound sources in full azimuth with three microphones. Estimating DOA with paired microphone arrays creates imaginary sound sources because of time delay of arrival (TDOA) being identical between real and imaginary sources. Imaginary sound sources can create chronic problems in multiple Sound Source Localization (SSL), because they can be localized as real sound sources. Our proposed approach is based on the observation that each microphone array creates imaginary sound sources, but the DOA of imaginary sources may be different depending on the orientation of the paired microphone array. With the fact that a real source would always be localized in the same direction regardless of the array orientation, we can suppress the imaginary sound sources by minimum filtering based on Steered Response Power — Phase Transform (SRP-PHAT) method. A set of experiments conducted in a real noisy environment showed that the proposed method was accurate in localizing multiple sound sources.

In this paper, we address the multiple sound source localization problem using time differences of arrival (TDOAs) of sound sources to a microphone array. Typically, TDOAs are estimated based on the peak extraction of the generalized crosscorrelation function. In multi-source cases, for any given microphone pair, it is hard to tell the correspondence between the sound sources and the extracted peaks. In this work, we develop a novel localization approach based on data association which combines multiple TDOAs from the same source across different microphone pairs. Firstly, the generalized cross correlation-phase transform (GCC-PHAT) function is evaluated and multiple peaks of the GCC function indicating candidate TDOAs are extracted for each pair of microphones. Next, we employ the multi-dimensional assignment algorithm to associate multiple TDOAs from the same source. Finally, multiple sound source localization is carried out based on the obtained TDOA associations across different microphone pairs. Experimental results show the proposed method achieves superior performance for multiple sound source localization compared to the competing algorithm, especially in noisy environments.

This paper presents a two-microphone technique for localization of multiple sound sources. Its fundamental structure is adopted from a binaural signal-processing scheme employed in biological systems for the localization of sources using interaural time differences (ITD). The two input signals are transformed to the frequency domain and analyzed for coincidences along left/right-channel delay-line pairs. The coincidence information is enhanced by a nonlinear operation followed by a temporal integration. The azimuths of the sound sources are estimated by integrating the coincidence locations across the broadband of frequencies in speech signals (the "direct" method). Further improvement is achieved by using a novel "stencil" filter pattern recognition procedure. This includes coincidences due to phase delays of greater than 2pi, which are generally regarded as ambiguous information. It is demonstrated that the stencil method can greatly enhance localization of lateral sources over the direct method. Also discussed and analyzed are two limitations involved in both methods, namely missed and artifactual sound sources. Anechoic chamber tests as well as computer simulation experiments showed that the signal-processing system generally worked well in detecting the spatial azimuths of four or six simultaneously competing sound sources.

Synchronistical localization, separation, and reconstruction for multiple sound sources are usually necessary in various situations, such as in conference rooms, living rooms, and supermarkets. To improve the intelligibility of speech signals, the application of deep neural networks (DNNs) has achieved considerable success in the area of time-domain signal separation and reconstruction. In this paper, we propose a hybrid microphone array signal processing approach for the nearfield scenario that combines the beamforming technique and DNN. Using this method, the challenge of identifying both the sound source location and content can be overcome. Moreover, the use of a sequenced virtual sound field reconstruction process enables the proposed approach to be quite suitable for a sound field which contains a dominant, stronger sound source and masked, weaker sound sources. Using this strategy, all traceable, mainly sound, sources can be discovered by loops in a given sound field. The operational duration and accuracy of localization are further improved by substituting the broadband weighted multiple signal classification (BW-MUSIC) method for the conventional delay-and-sum (DAS) beamforming algorithm. The effectiveness of the proposed method for localizing and reconstructing speech signals was validated by simulations and experiments with promising results. The localization results were accurate, while the similarity and correlation between the reconstructed and original signals was high.

It is very important to capture distant-talking speech for a hands-free speech interface with high quality. A microphone array is an ideal candidate for this purpose. However, this approach requires localizing the target talker. Conventional talker localization algorithms in multiple sound source environments not only have difficulty localizing the multiple sound sources accurately, but also have difficulty localizing the target talker among known multiple sound source positions. To cope with these problems, we propose a new talker localization algorithm consisting of two algorithms. One is DOA (direction of arrival) estimation algorithm for multiple sound source localization based on CSP (cross-power spectrum phase) coefficient addition method. The other is statistical sound source identification algorithm based on GMM (Gaussian mixture model) for localizing the target talker position among localized multiple sound sources. In this paper, we particularly focus on the talker localization performance based on the combination of these two algorithms with a microphone array. We conducted evaluation experiments in real noisy reverberant environments. As a result, we confirmed that multiple sound signals can be identified accurately between ‘‘speech’’ or ‘‘non-speech’’ by the proposed algorithm. [Work supported by ATR, and MEXT of Japan.]

For a hands-free speech interface, it is very important to capture distant talking speech with high quality. A microphone array is an ideal candidate for this purpose. However, this approach requires localizing the target talker. Conventional talker localization algorithms in multiple sound source environments not only have difficulty localizing the multiple sound sources accurately, but also have difficulty localizing the target talker among known multiple sound source positions. To cope with these problems, we propose a new talker localization algorithm consisting of two algorithms. One is DOA (Direction Of Arrival) estimation algorithm for multiple sound source localization based on CSP (Cross-power Spectrum Phase) coefficient addition method. The other is statistical sound source identification algorithm based on GMM (Gaussian Mixture Model) for localizing the target talker position among localized multiple sound sources. In this paper, we particularly focus on the talker localization performance based on the combination of these two algorithms with a microphone array.

In the microphone array, the phase error of each microphone causes a deviation in sound source localization. At present, there is a lack of effective methods for phase error calibration of the entire microphone array. In order to solve this problem, a phase mismatch calculation method based on multiple sound sources is proposed. This method requires collecting data from multiple sound sources in turn, and constructing a nonlinear equation setthrough the signal delay and the geometric relationship between the microphones and the sound source positions. The phase mismatch of each microphone can be solved from the nonlinear equation set. Taking the single frequency signal as an example, the feasibility of the method is verified by experiments in a semi-anechoic chamber. The phase mismatches are compared with the calibration results of exchanging microphone. The difference of the phase error values measured by the two methods is small. The experiment also shows that the accuracy of sound source localization by beamforming is improved. The method is efficient for phase error calibration of arrays with a large number of microphones.

Mobile robots communicating with people would benefit from being able to detect sound sources to help localize interesting events in real-life settings. We propose using a spherical robot with four microphones to determine the spatial locations of multiple sound sources in ordinary rooms. The arrival temporal disparities from phase difference histograms are used to calculate the time differences. A precedence effect model suppresses the influence of echoes in reverberant environments. To integrate spatial cues of different microphones, we map the correlation between different microphone pairs on a 3D map corresponding to the azimuth and elevation of sound source direction. Results of experiments indicate that our proposed system provides sound source distribution very clearly and precisely, even concurrently in reverberant environments with the Echo Avoidance (EA) model.

Sound source localization is the problem of estimating the positions of one or several sound sources. In terms of binaural audio, localization is a paramount perceptual characteristic which can be assessed subjectively or objectively. For objective evaluation of binaural sound localization, typical methods exploit binaural or monaural cues to predict directions of sound sources. Since multiple sound sources are often perceived simultaneously in daily sound scenes, an objective sound localization model which can detect temporally overlapping sources is required. In this paper, we propose a binaural multiple sound source localization network (BMSSLnet) model, which can predict framewise azimuths without prior knowledge of sound source number in a binaural audio signal. We implement multiple azimuth prediction as a multi-label classification task, and propose to use separated multi-label cross-entropy and mean square error as the loss function. Experimental results show that the proposed model obtains the average precision of 0.9 and 0.75 for spatial prediction on the anechoic dataset and reverberant dataset with up to three temporally overlapping sources, respectively. Framewise temporal prediction with average accuracy of 38.3 ms is achieved.

Deep learning-based method for multiple sound source localization with high resolution and accuracy

Accurate localization of sound sources is indispensable for high‐quality sound capture. The difference of distance from a sound source to two microphones causes a time delay of arrival (TDOA). The cross‐power spectrum phase analysis (CSP) method has been proposed to estimate the TDOA based on correlation in signals. The sound source can be localized theoretically as a crossing point of sound directions estimated using two pairs of microphones. However, in a real environment, microphones receive sounds that have originated through reverberation and reflection. This situation makes TDOA estimation difficult, and accurate sound localization is difficult when using only two pairs of microphones. The authors propose a new method of sound localization based on multiple microphones that are widely distributed and placed under the ceiling of a room. A number of pairs of microphones give a series of CSP correlation coefficients as a function of the time delay. The CSP coefficients for hypothetical sound sources are accumulated over all the microphone pairs. The number of microphones is 16. The sound source is localized as a hypothesis that maximizes the accumulated CSP coefficient. Experiment results obtained in a real room show that the proposed method improves the localization accuracy.

This paper presents a new approach of microphone array sampling and processing for acoustic source localization. By sampling circular arrays in a round robin fashion, nonlinear modulations are purposefully induced by means of the Doppler effect. The discrete time Teager-Kaiser Energy Operator is then used to analyze these modulations. It enables a batch-based localization of multiple sound sources at low complexity. The proposal system uses a small circular array but is suitable for other array geometries as well. In contrast to cross-correlation based localization techniques, we process only two signals while we maintain a circular symmetric system with no preferred look direction. Experiments are reported for up to 5 simultaneously active speech sources.

Sound source localization based on a microphone array can be used in telephone or video conference systems to detect the speaker's location. A method based on time difference of arrival (TDOA) estimation is an ideal candidate as an effective method for sound source localization. Time delay estimation (TDE) and source localization are two main factors of it. The crosspower-spectrum phase (CSP) method is a popular TDE technique in sound source localization based on microphone arrays. The relatively precise TDE can be achieved by using the CSP method in the presence of weak noise and reverberation. The performance degrades dramatically in the case of low signal-to-noise ratio (SNR) and strong reverberation. A generalized CSP (GCSP) method for TDE is proposed to improve the performance of CSP. The weighting function of GCSP is estimated from the multi-channel signals of the microphone array. Simulation results show that the proposed method outperforms the CSP method.

Various microphone array geometries (e.g., linear, circular, square, cubic, spherical, etc.) have been used to improve the positioning accuracy of sound source localization. However, whether these array structures are optimal for various specific localization scenarios is still a subject of debate. This paper addresses a microphone array optimization method for sound source localization based on TDOA (time difference of arrival). The geometric structure of the microphone array is established in parametric form. A triangulation method with TDOA was used to build the spatial sound source location model, which consists of a group of nonlinear multivariate equations. Through reasonable transformation, the nonlinear multivariate equations can be converted to a group of linear equations that can be approximately solved by the weighted least square method. Then, an optimization model based on particle swarm optimization (PSO) algorithm was constructed to optimize the geometric parameters of the microphone array under different localization scenarios combined with the spatial sound source localization model. In the optimization model, a reasonable fitness evaluation function is established which can comprehensively consider the positioning accuracy and robustness of the microphone array. In order to verify the array optimization method, two specific localization scenarios and two array optimization strategies for each localization scenario were constructed. The optimal array structure parameters were obtained through numerical iteration simulation. The localization performance of the optimal array structures obtained by the method proposed in this paper was compared with the optimal structures proposed in the literature as well as with random array structures. The simulation results show that the optimized array structure gave better positioning accuracy and robustness under both specific localization scenarios. The optimization model proposed could solve the problem of array geometric structure design based on TDOA and could achieve the customization of microphone array structures under different specific localization scenarios.

The estimation of distance and power of multiple sound sources by combining three-dimensional sound intensity and beamforming