MIMO Noise Suppression Preserving Spatial Cues for Sound Source Localization in Mobile Robot

In this position paper, a fast real-time algorithm for the sound source localization system using microphone array is proposed. The algorithm includes three steps: Delay time estimation using time difference of arrival (TDOA); Searching area selection based on angle of arrival; and, Sound source localization using a newly golden section searching (GSS) method. Thanks to the first two steps, we can narrow down the searching area significantly. Then, the real-time sound source localization is achieved by the effective GSS method. Finally, satisfactory simulation and experimental results are illustrated.

It is very important to capture distant-talking speech for a hands-free speech interface with high quality. A microphone array is an ideal candidate for this purpose. However, this approach requires localizing the target talker. Conventional talker localization algorithms in multiple sound source environments not only have difficulty localizing the multiple sound sources accurately, but also have difficulty localizing the target talker among known multiple sound source positions. To cope with these problems, we propose a new talker localization algorithm consisting of two algorithms. One is DOA (direction of arrival) estimation algorithm for multiple sound source localization based on CSP (cross-power spectrum phase) coefficient addition method. The other is statistical sound source identification algorithm based on GMM (Gaussian mixture model) for localizing the target talker position among localized multiple sound sources. In this paper, we particularly focus on the talker localization performance based on the combination of these two algorithms with a microphone array. We conducted evaluation experiments in real noisy reverberant environments. As a result, we confirmed that multiple sound signals can be identified accurately between ‘‘speech’’ or ‘‘non-speech’’ by the proposed algorithm. [Work supported by ATR, and MEXT of Japan.]

For a hands-free speech interface, it is very important to capture distant talking speech with high quality. A microphone array is an ideal candidate for this purpose. However, this approach requires localizing the target talker. Conventional talker localization algorithms in multiple sound source environments not only have difficulty localizing the multiple sound sources accurately, but also have difficulty localizing the target talker among known multiple sound source positions. To cope with these problems, we propose a new talker localization algorithm consisting of two algorithms. One is DOA (Direction Of Arrival) estimation algorithm for multiple sound source localization based on CSP (Cross-power Spectrum Phase) coefficient addition method. The other is statistical sound source identification algorithm based on GMM (Gaussian Mixture Model) for localizing the target talker position among localized multiple sound sources. In this paper, we particularly focus on the talker localization performance based on the combination of these two algorithms with a microphone array.

Multiple sound source localization is one of the important topic in speech processing. GCC function is used as a traditional algorithm for sound source localization. This function estimates DOA for multiple speakers by calculation the cross-correlation between microphone signals but its accuracy decreases in adverse conditions. The aim of proposed method in this paper is localization of multiple simultaneous speakers in undesirable condition. The proposed method is based on novel 3D nested microphone array in combination with obtained information of Discrete Wavelet Transform (DWT) and subband processing. The proposed 3D nested microphone array prepares the condition for 3D localization and eliminates the spatial aliasing between microphone signals. Also, we propose the DWT for extraction the information of speech signal. Since, the spectral information of speech signal concentrates on low frequencies, we propose a structure of filter bank based on DWT to increase the frequency resolution on low frequencies. The performed evaluation on real and simulated data shows the superiority of our proposed method in comparison with Fullband and subband processing with uniform filters and uniform microphone array.

Many kinds of sound source localization systems have been developed for detecting a direction of sound source. They are commonly using time delay of arrival (TDOA) or interaural time difference (ITD) algorithm for sound source localization where, especially, the ITD is the difference in arrival time of a sound between two ears. It is largely changed depending on frequency components of sound even though the sound source is located in the same place. In this paper we propose a binaural sound localization system using sparse coding based ITD (SITD) and self-organizing map (SOM). The sparse coding is used for decomposing given sounds into three components: time, frequency and magnitude. Moreover we estimate the azimuth angle through the SOM. This localization system is installed in our robot that has two ears, head and body. We use PeopleBot as a body of the robot.

In this paper we present a novel signal processing algorithm and array for sound source localization in an enclosed area. This method, which has some similarity to the human eye structure, consists of a novel hemispherical microphone array with microphones on the shell and one microphone in the sphere center. A signal processing scheme utilizes parallel creation of a special closeness functions for each microphone direction on the shell. The closeness functions have output values that are linearly proportional to spatial angular difference between the sound source direction and each of the shell microphone directions, in close vicinity of them. Finally by choosing directions corresponding to the highest closeness function values and implementing a linear weighted spatial averaging on those directions we estimate the sound source direction. Contrary to traditional algorithmic sound source localization techniques, our method is based on some simple parallel mathematical calculations in the time domain; therefore it can be easily implemented on a custom designed integrated circuit

Chapter 15 - A novel sound source localization method using a global-best guided cuckoo search algorithm for drone-based search and rescue operations

Rescue robots have been developed for search and rescue operations in times of large-scale disasters. Such a robot is used to search for survivors in disaster sites by capturing their voices with its microphone array. However, since the robot has many vibration motors, ego noise is mixed with voices, and it is difficult to differentiate the ego noise from a call for help from a disaster survivor. In our previous works, an ego noise reduction technique that combines a method of blind source separation called independent low-rank matrix analysis and postprocessing for noise cancellation was proposed. In the practical use of this robot, to determine the precise location of survivors, the direction of the observed voice should be estimated after the ego noise reduction process. To achieve this objective, in this study, a new hose-shaped rescue robot with microphone arrays was developed. Moreover, we adapt postfilter called MOSIE to our previous noise reduction method to listen to stereo sound because this robot can record stereo sound. By performing in a simulated disaster site, we confirm that the operator can perceive the direction of a survivor's location by applying a speech enhancement technique combining independent low-rank matrix analysis, noise cancellation, and postfiltering to the observed multichannel noisy signals.

For the development and evaluation of a sound source localization and separation methods, a concise audio dataset with complete geometrical information about the room, the positions of the sound sources, and the array of microphones is needed. Computer simulation of such audio and geometrical data often relies on simplifications and are sufficiently accurate only for a specific set of conditions. It is generally desired to evaluate algorithms on real-world data. For a three-dimensional sound source localization or direction of arrival estimation, a non-coplanar microphone array is needed.Simplest and most general type of non-coplanar array is a tetrahedral array. There is a lack of openly accessible realworld audio datasets obtained using such arrays. We present an audio dataset for the evaluation of sound source localization algorithms, which involve tetrahedral microphone arrays. The dataset is complete with the geometrical information of the room, the positions of the sound sources and the microphone array. Array audio data was captured for two tetrahedral microphone arrays with different distances between microphones and one or two active sound sources. The dataset is suitable for speech recognition and direction-of-arrival estimation, as the signals used for sound sources were speech signals.

Aiming at the leakage problem of ship pipeline system, this paper proposes a leak detection method of ship pipeline system, which is based on video target detection and tracking algorithm and sound source localization algorithm of microphone array, and uses the image information collected by camera and audio information collected by microphone array for multimodal information fusion to improve the robustness of sound source localization system and the accuracy of leak detection results. In this paper, the whole audio-video processing method is introduced theoretically, the video detection and tracking module adopts YOLOv5m algorithm as the framework of visual detection, the source localization module adopts cross-type microphone array to obtain audio information, and the time delay of each microphone received signal is combined to calculate the sound source orientation, and finally the system software and hardware design are carried out, compared with the single-modal algorithm and system, this system has higher tracking accuracy and performance compared to a single modal algorithm and system.

This paper presents a new constrained subband beamforming algorithm to enhance speech signals generated by a moving source in a noisy environment. The beamformer is based on the principle of a soft constraint defined for a specified region corresponding to a known source location. The soft constraint secures the spatial-temporal passage of the desired source signal in the adaptive update of the beamforming weights and guarantees the full rank property of the matrix inverted in the update. The source of interest is modelled as a cluster of stationary point sources and source motion is accommodated by revising the point source cluster. The source modelling and its direct exploitation in the beamformer through covariance estimates are presented. An algorithm for sound source localization is used for speaker movement tracking and this information is exploited to update the spatial distribution in the source model. Evaluation in a real environment with a moving speaker shows a significant noise and hands-free interference suppression within the conventional telephone bandwidth. This is achieved with a negligible impact on speech distortion.

Sound Source Localization (SSL) is an essential function for robot audition and yields the location and number of sound sources, which are utilized for post-processes such as sound source separation. SSL for a robot in a real environment mainly requires noise-robustness, high resolution and real-time processing. A technique using microphone array processing, that is, Multiple Signal Classification based on Standard EigenValue Decomposition (SEVD-MUSIC) is commonly used for localization. We improved its robustness against noise with high power by incorporating Generalized EigenValue Decomposition (GEVD). However, GEVD-based MUSIC (GEVD-MUSIC) has mainly two issues: 1) the resolution of pre-measured Transfer Functions (TFs) determines the resolution of SSL, 2) its computational cost is expensive for real-time processing. For the first issue, we propose a TF interpolation method integrating time-domain-based and frequency-domain-based interpolation. The interpolation achieves super-resolution SSL, whose resolution is higher than that of the pre-measured TFs. For the second issue, we propose two methods, MUSIC based on Generalized Singular Value Decomposition (GSVD-MUSIC), and Hierarchical SSL (H-SSL). GSVD-MUSIC drastically reduces the computational cost while maintaining noise-robustness in localization. H-SSL also reduces the computational cost by introducing a hierarchical search algorithm instead of using greedy search in localization. These techniques are integrated into an SSL system using a robot embedded microphone array. The experimental result showed: the proposed interpolation achieved approximately 1 degree resolution although we have only TFs at 30 degree intervals, GSVD-MUSIC attained 46.4% and 40.6% of the computational cost compared to SEVD-MUSIC and GEVD-MUSIC, respectively, H-SSL reduces 59.2% computational cost in localization of a single sound source.

This paper proposes a auditory system based on microphone array and composed of three parts: noise suppression, sound source localization (SSL) and beamforming. The ego-noise is harmful to the auditory system embedded in a small robot and very hard to be suppressed. We have proposed a method to design active acoustic metamaterials (AAMMs) that perform extremely well in the ego-noise suppression in a small robot. In this case, our research mainly focuses on the issues of reliability and real time on auditory system algorithms. A 2-stage SSL algorithm is adopted in our system to reduce the complexity. Furthermore, the matrix inversion can be processed in parallel with signal sampling, which means very low latency between the output and the input.

A growing interest in microphone arrays technology has been observed in the automotive industry, e.g. for the development of immersive infotainment systems or Active Noise Control (ANC) systems. However, the human presence always limits the usage of microphone arrays in driving conditions at driver seat. The latter being often the most important position of the car cabin, a wearable microphone array is particularly interesting. In this paper, a wearable helmet microphone array is presented: it features 32 microphones arranged over the surface of a helmet, which also integrates a specifically designed Analog-to-Digital (A/D) converter, delivering digital signals over the Automotive Audio Bus (A2B). Digital signals are collected using a control unit located in the passenger compartment, that can either deliver digital signals to a personal computer or analog signals to an external acquisition system, by means of Digital-to-Analog (D/A) converters. A prototype was built and acoustically characterized to calculate the beamforming matrix required to convert the recordings (pressure signals) into Ambisonics signals (a spatial audio format). The proposed solution was compared to the reference spherical microphone array of the last decade, demonstrating a much better performance in sound source localization at low frequencies, where ANC systems are mainly effective.

With the development of artificial intelligence voice technology, array signal processing technology has been widely used in intelligent human–computer interaction. In this field, the sound source localization technology of the microphone array system is also one of the key technologies in the composition of intelligent systems, and it is also a research hotspot technology, which is of great significance for improving human–computer interaction ability. In this paper, the microphone array technology is used to demonstrate and improve the sound source localization of indoor speech signals. A spatial distributed microphone array localization algorithm is proposed to improve the accuracy of the indoor voice source signal localization in the presence of indoor noise and reverberation.