Spatial Localization of Concurrent Multiple Sound Sources Using Phase Candidate Histogram

Accurate localization of multiple sound sources is indispensable for the microphone array-based high quality sound capture. For single sound source localization, the CSP (cross-power spectrum phase analysis) method has been proposed. The CSP method localizes a sound source as a crossing point of sound directions estimated using different microphone pairs. However, when localizing multiple sound sources, the CSP method has a problem that the localization accuracy is degraded due to cross-correlation among different sound sources. To solve this problem, this paper proposes a new method which suppresses the undesired cross-correlation by synchronous addition of CSP coefficients derived from multiple microphone pairs. Experiment results in a real room showed that the proposed method improves the localization accuracy when increasing the number of the synchronous addition.

In this paper, we address the multiple sound source localization problem using time differences of arrival (TDOAs) of sound sources to a microphone array. Typically, TDOAs are estimated based on the peak extraction of the generalized crosscorrelation function. In multi-source cases, for any given microphone pair, it is hard to tell the correspondence between the sound sources and the extracted peaks. In this work, we develop a novel localization approach based on data association which combines multiple TDOAs from the same source across different microphone pairs. Firstly, the generalized cross correlation-phase transform (GCC-PHAT) function is evaluated and multiple peaks of the GCC function indicating candidate TDOAs are extracted for each pair of microphones. Next, we employ the multi-dimensional assignment algorithm to associate multiple TDOAs from the same source. Finally, multiple sound source localization is carried out based on the obtained TDOA associations across different microphone pairs. Experimental results show the proposed method achieves superior performance for multiple sound source localization compared to the competing algorithm, especially in noisy environments.

This paper describes a method for estimating Direction Of Arrival (DOA) of multiple sound sources in full azimuth with three microphones. Estimating DOA with paired microphone arrays creates imaginary sound sources because of time delay of arrival (TDOA) being identical between real and imaginary sources. Imaginary sound sources can create chronic problems in multiple Sound Source Localization (SSL), because they can be localized as real sound sources. Our proposed approach is based on the observation that each microphone array creates imaginary sound sources, but the DOA of imaginary sources may be different depending on the orientation of the paired microphone array. With the fact that a real source would always be localized in the same direction regardless of the array orientation, we can suppress the imaginary sound sources by minimum filtering based on Steered Response Power — Phase Transform (SRP-PHAT) method. A set of experiments conducted in a real noisy environment showed that the proposed method was accurate in localizing multiple sound sources.

Sound source localization is the problem of estimating the positions of one or several sound sources. In terms of binaural audio, localization is a paramount perceptual characteristic which can be assessed subjectively or objectively. For objective evaluation of binaural sound localization, typical methods exploit binaural or monaural cues to predict directions of sound sources. Since multiple sound sources are often perceived simultaneously in daily sound scenes, an objective sound localization model which can detect temporally overlapping sources is required. In this paper, we propose a binaural multiple sound source localization network (BMSSLnet) model, which can predict framewise azimuths without prior knowledge of sound source number in a binaural audio signal. We implement multiple azimuth prediction as a multi-label classification task, and propose to use separated multi-label cross-entropy and mean square error as the loss function. Experimental results show that the proposed model obtains the average precision of 0.9 and 0.75 for spatial prediction on the anechoic dataset and reverberant dataset with up to three temporally overlapping sources, respectively. Framewise temporal prediction with average accuracy of 38.3 ms is achieved.

This paper presents a two-microphone technique for localization of multiple sound sources. Its fundamental structure is adopted from a binaural signal-processing scheme employed in biological systems for the localization of sources using interaural time differences (ITD). The two input signals are transformed to the frequency domain and analyzed for coincidences along left/right-channel delay-line pairs. The coincidence information is enhanced by a nonlinear operation followed by a temporal integration. The azimuths of the sound sources are estimated by integrating the coincidence locations across the broadband of frequencies in speech signals (the "direct" method). Further improvement is achieved by using a novel "stencil" filter pattern recognition procedure. This includes coincidences due to phase delays of greater than 2pi, which are generally regarded as ambiguous information. It is demonstrated that the stencil method can greatly enhance localization of lateral sources over the direct method. Also discussed and analyzed are two limitations involved in both methods, namely missed and artifactual sound sources. Anechoic chamber tests as well as computer simulation experiments showed that the signal-processing system generally worked well in detecting the spatial azimuths of four or six simultaneously competing sound sources.

Synchronistical localization, separation, and reconstruction for multiple sound sources are usually necessary in various situations, such as in conference rooms, living rooms, and supermarkets. To improve the intelligibility of speech signals, the application of deep neural networks (DNNs) has achieved considerable success in the area of time-domain signal separation and reconstruction. In this paper, we propose a hybrid microphone array signal processing approach for the nearfield scenario that combines the beamforming technique and DNN. Using this method, the challenge of identifying both the sound source location and content can be overcome. Moreover, the use of a sequenced virtual sound field reconstruction process enables the proposed approach to be quite suitable for a sound field which contains a dominant, stronger sound source and masked, weaker sound sources. Using this strategy, all traceable, mainly sound, sources can be discovered by loops in a given sound field. The operational duration and accuracy of localization are further improved by substituting the broadband weighted multiple signal classification (BW-MUSIC) method for the conventional delay-and-sum (DAS) beamforming algorithm. The effectiveness of the proposed method for localizing and reconstructing speech signals was validated by simulations and experiments with promising results. The localization results were accurate, while the similarity and correlation between the reconstructed and original signals was high.

Deep learning-based method for multiple sound source localization with high resolution and accuracy

It is very important to capture distant-talking speech for a hands-free speech interface with high quality. A microphone array is an ideal candidate for this purpose. However, this approach requires localizing the target talker. Conventional talker localization algorithms in multiple sound source environments not only have difficulty localizing the multiple sound sources accurately, but also have difficulty localizing the target talker among known multiple sound source positions. To cope with these problems, we propose a new talker localization algorithm consisting of two algorithms. One is DOA (direction of arrival) estimation algorithm for multiple sound source localization based on CSP (cross-power spectrum phase) coefficient addition method. The other is statistical sound source identification algorithm based on GMM (Gaussian mixture model) for localizing the target talker position among localized multiple sound sources. In this paper, we particularly focus on the talker localization performance based on the combination of these two algorithms with a microphone array. We conducted evaluation experiments in real noisy reverberant environments. As a result, we confirmed that multiple sound signals can be identified accurately between ‘‘speech’’ or ‘‘non-speech’’ by the proposed algorithm. [Work supported by ATR, and MEXT of Japan.]

The estimation of distance and power of multiple sound sources by combining three-dimensional sound intensity and beamforming

For a hands-free speech interface, it is very important to capture distant talking speech with high quality. A microphone array is an ideal candidate for this purpose. However, this approach requires localizing the target talker. Conventional talker localization algorithms in multiple sound source environments not only have difficulty localizing the multiple sound sources accurately, but also have difficulty localizing the target talker among known multiple sound source positions. To cope with these problems, we propose a new talker localization algorithm consisting of two algorithms. One is DOA (Direction Of Arrival) estimation algorithm for multiple sound source localization based on CSP (Cross-power Spectrum Phase) coefficient addition method. The other is statistical sound source identification algorithm based on GMM (Gaussian Mixture Model) for localizing the target talker position among localized multiple sound sources. In this paper, we particularly focus on the talker localization performance based on the combination of these two algorithms with a microphone array.

In the microphone array, the phase error of each microphone causes a deviation in sound source localization. At present, there is a lack of effective methods for phase error calibration of the entire microphone array. In order to solve this problem, a phase mismatch calculation method based on multiple sound sources is proposed. This method requires collecting data from multiple sound sources in turn, and constructing a nonlinear equation setthrough the signal delay and the geometric relationship between the microphones and the sound source positions. The phase mismatch of each microphone can be solved from the nonlinear equation set. Taking the single frequency signal as an example, the feasibility of the method is verified by experiments in a semi-anechoic chamber. The phase mismatches are compared with the calibration results of exchanging microphone. The difference of the phase error values measured by the two methods is small. The experiment also shows that the accuracy of sound source localization by beamforming is improved. The method is efficient for phase error calibration of arrays with a large number of microphones.

Sound source separation is a hot topic in array signal processing. In fact, it is difficult to separate multiple sound sources in reverberant environments. To solve this problem, a two-stage multiple sound source separation method is proposed based on the fully-convolutional time-domain audio separation network (Conv-TasNet) and deep neural network (DNN). In the first stage, the end-to-end separation network (i.e., the Conv-TasNet) is employed to separate the sound sources from the signal recorded in reverberant environments. We use the encoder to generate the time domain representation of the recorded signal, and the mask is obtained by the Conv-TasNet model. The recovery of each separated sound source signal is conducted by multiplying the output of the encoder by the mask. In the second stage, the separated signal is enhanced with the single DNN. The training target of the DNN is to obtain the ideal enhancement mask, which is calculated by integrating the amplitude of the frequency domain coefficients of both separated signal and clean signal. The amplitude of the frequency domain coefficients of the separated signals is used as the input of DNN in order to predict the ideal ratio mask (IRM). IRM is multiplied by the original amplitude to obtain the enhanced amplitude of the frequency domain coefficients, which is integrated with phase to obtain the enhanced source signal. The results of the subjective and objective evaluation show that, compared with the reference methods, the proposed method achieves better separation quality in both reverberant and anechoic acoustic environments.

How to improve the performance of multiple sound source localization method, this paper studies the parameter and process design in multiple sound source position. First, it divides a signal into several sub-frames, and it calculates signal variation; then it uses the clustering algorithm to divide the estimated values into several categories; finally it calculates the final sound source location estimation. Experimental results show that the positioning algorithm is effective. There are many parameters will affect the gunfire correct rate and error rate, the parameters and angle selection will affect sound source position estimation.

This paper assesses the 3D sound IP phone service (3D IP phone) developed by the authors under environment with multiple sound sources by experiment. The assessment consists of evaluation of the sound source localization, subjective evaluation and objective one. The authors utilize two sound sources that are conversations carried on by a man and a woman. From the experimental results, this paper shows that multiple sound sources can promote listener's reproduction of sound images. This means that the proposed system can give the higher feeling of being in more real environment.

This paper presents an automatic sound source localization approach based on a combination of the basic time delay estimation sub-methods namely, Time Difference of Arrival (TDOA), and Steered Response Power (SRP) methods. The TDOA method is a fast but vulnerable approach for finding the sound source location in long distances and reverberant environments and is so sensitive in noisy situations. On the other hand, the conventional SRP method is time consuming, but a successful approach to accurately find sound source location in noisy and reverberant environments. Also, another SRP-based method, SRP Phase Transform (SRP-PHAT), has been suggested for the better noise robustness and more accuracy of sound source localization. In this paper, based on the combination of TDOA and SRP based methods, two approaches were proposed for sound source localization. In the first proposed approach called Classical TDOA-SRP, the TDOA method is used to find the approximate sound source direction and then SRP based methods were used to find the accurate location of sound source in the Field of View (FOV) which is obtained by the TDOA method. In the second proposed approach which called Optimal TDOA-SRP, for more reduction of computational processing time of SRP-based methods and better noise robustness, a new criterion has been proposed for finding the effective FOV which is obtained through the TDOA method. Experiments were carried out under different conditions confirming the validity of the purposed approaches.