Microphone Array Acoustic Source Localization system based on Deep Learning

This paper presents a novel localization method that leverages two sets of distributed microphone arrays using the Generalized Cross-Correlation Phase Transform (GCC-PHAT) technique to improve the performance of anti-drone systems. In contrast to conventional sound source localization techniques, the proposed approach enhances localization accuracy by precisely estimating the azimuth angle while considering the unique acoustic characteristics of drones. The effectiveness of the proposed method was validated through both simulations and field tests. Simulation results revealed that, in ideal channel conditions, the proposed method significantly reduced the mean and variance of localization errors compared to existing techniques, resulting in more accurate positioning. Furthermore, in noisy environments, the proposed approach consistently outperformed the comparison method across various Signal-to-Noise Ratio (SNR) levels, achieving up to 2.13 m of improvement at SNR levels above 0 dB. While the comparison method exhibited decreased localization accuracy along the y-axis and z-axis, the proposed method maintained stable performance across all axes by effectively distinguishing between azimuth and elevation angles. Field test results closely mirrored the simulation outcomes, further confirming the robustness and reliability of the proposed localization approach.

Acoustic bird species classification under low SNR and small-scale dataset conditions

Sound source localization based on discrimination of cross-correlation functions

We focus on the sound source localization (SSL) problem of small/weak voices recorded by small humanoid robots, such as Nao. The localization of small voice on such robots is more difficult than those with large bodies because 1) the number of microphones and their positions are restricted and 2) the signal to noise ratio (SNR) between internal noise and human speech is low. Multiple Signal Classification (MUSIC) is a promising noise-robust SSL method, and it has several variants in its eigenvector decomposition process, such as generalized eigenvalue decomposition (GEVD-MUSIC) and generalized singular value decomposition (GSVD-MUSIC). However, their performances are seriously affected by noise properties, the number of microphone, and their configurations on each robot's body. Since these have been confirmed only on robots with large bodies and many microphones, we need to investigate which variants perform better on a small humanoid robot with internal noise in it. We furthermore propose another MUSIC based on transformed steering vector (TSV-MUSIC) as a new implementation of GEVD-MUSIC. The computational cost of TSV-MUSIC is reduced compared with GEVD-MUSIC by changing its matrix multiplication procedures. Experimental results using real recoded data showed that our TSV-MUSIC outperformed others in terms of localization correctness by about 10 points under low SNR condition. We also compared the properties and performances of MUSIC-based SSLs by using simulated data.

In distributed microphone arrays, each microphone is connected to a separate recording device, which requires compensation for their different clock shifts. In this paper, we propose a new type of distributed arrays, in which the microphones are moving. This mobility introduces innovation that can be used to estimate the positions of both the sound sources and the microphones, as well as compensate for the clock shifts. Estimation is done using extended Kaiman filters with heuristics that improve the convergence. We discuss the proposed method in the context of mobile robotics: each microphone is attached to a controllable mobile robot. In this scenario, the array can be used to localize sound-emitting objects in the robot's environment, while providing information about robots' positions. Results show that the error of estimating robots' positions is as low as 5 cm with the source source localization error of 6 mm and the clock synchronization error of 178 μs, for the case of 9 microphones and 5 sound sources.

Detection and classification of rotorcraft targets are of great significance not only in civil fields but also in defense. However, up to now, it is still difficult for the traditional radar signal processing methods to detect and distinguish rotorcraft targets from various types of moving objects. Moreover, it is even more challenging to classify different types of helicopters. As the development of high-precision radar, classification of moving targets by micro-Doppler features has become a promising research topic in the modern signal processing field. In this paper, we propose to use the deep convolutional neural networks (DCNNs) in rotorcraft detection and helicopter classification based on Doppler radar signals. We apply DCNN directly to raw micro-Doppler spectrograms for rotorcraft detection and classification. The proposed DCNNs can learn the features automatically from the micro-Doppler signals without introducing any domain background knowledge. Simulated data are used in the experiments. The experimental results show that the proposed DCNNs achieve superior accuracy in rotorcraft detection and superior accuracy in helicopter classification, outperforming the traditional radar signal processing methods.

In this paper, we address the multiple sound source localization problem using time differences of arrival (TDOAs) of sound sources to a microphone array. Typically, TDOAs are estimated based on the peak extraction of the generalized crosscorrelation function. In multi-source cases, for any given microphone pair, it is hard to tell the correspondence between the sound sources and the extracted peaks. In this work, we develop a novel localization approach based on data association which combines multiple TDOAs from the same source across different microphone pairs. Firstly, the generalized cross correlation-phase transform (GCC-PHAT) function is evaluated and multiple peaks of the GCC function indicating candidate TDOAs are extracted for each pair of microphones. Next, we employ the multi-dimensional assignment algorithm to associate multiple TDOAs from the same source. Finally, multiple sound source localization is carried out based on the obtained TDOA associations across different microphone pairs. Experimental results show the proposed method achieves superior performance for multiple sound source localization compared to the competing algorithm, especially in noisy environments.

We propose a multimodal speaker segmentation algo-rithm with two main contributions: First, we suggest a hidden Markov model architecture that performs fusion of the three modalities: a multi-camera system for participant localization, a microphone array for speaker localization, and a speaker identification system; Second, we present a novel method for dealing with overlapped speech segments through a likelihood model of the microphone array obser-vations that uses multiple local maxima of the Steered Power Response Generalized Cross Correlation Phase Transform (SPR-GCC-PHAT) function in the Joint Probabilistic Data Association (JPDA) framework. Results show that the pro-posed method outperforms standard speaker segmentation systems based on: (a) speaker identification and; (b) mi-crophone array processing, for datasets with the significant portion (27.4%) of overlapped speech, and scores as high as 94.4 % on the F-measure scale. 1

Bogies are an important component of high-speed trains. The level of mechanical performance of bogies has a major influence on the safety and reliability of high-speed train. Therefore, conducting fault diagnoses on bogies with big data is very important. Fault mechanisms of bogies are very complex, and feature signals are nonobvious. For these reasons, fault information of bogies cannot be effectively extracted using the traditional signal processing method. Therefore, this paper adopted the deep neural network to recognize faults in bogies. The deep neural network offers numerous benefits in this context. Using deep neural networks, fault information in a signal spectrum can be extracted in a selfadaptive method. This technique is free of dependence on extensive signal processing knowledge and diagnostic experience. Compared with the traditional intelligent diagnosis method, the deep neural network can obtain a higher diagnostic accuracy. Additionally, the deep neural network does not depend on the sample size, and it can obtain high diagnostic accuracy even when the sample size is relatively small. It also achieves very high diagnostic accuracy applied to high-speed trains with different speeds and different faults, which shows that the method is extensively applicable. Furthermore, the recognition accuracy rate of the deep neural network under normal conditions can reach 100%. This method provides a new paradigm for fault diagnosis of the high-speed train with big data and plays an important role in this field.

Synchronistical localization, separation, and reconstruction for multiple sound sources are usually necessary in various situations, such as in conference rooms, living rooms, and supermarkets. To improve the intelligibility of speech signals, the application of deep neural networks (DNNs) has achieved considerable success in the area of time-domain signal separation and reconstruction. In this paper, we propose a hybrid microphone array signal processing approach for the nearfield scenario that combines the beamforming technique and DNN. Using this method, the challenge of identifying both the sound source location and content can be overcome. Moreover, the use of a sequenced virtual sound field reconstruction process enables the proposed approach to be quite suitable for a sound field which contains a dominant, stronger sound source and masked, weaker sound sources. Using this strategy, all traceable, mainly sound, sources can be discovered by loops in a given sound field. The operational duration and accuracy of localization are further improved by substituting the broadband weighted multiple signal classification (BW-MUSIC) method for the conventional delay-and-sum (DAS) beamforming algorithm. The effectiveness of the proposed method for localizing and reconstructing speech signals was validated by simulations and experiments with promising results. The localization results were accurate, while the similarity and correlation between the reconstructed and original signals was high.

Human activities in the marine environment coupled with the efficient propagation of acoustic waves, have significantly increased ocean noise levels. This rise poses a threat to marine biodiversity, especially impacting vocal-dependent marine mammals. Therefore, obtaining a more comprehensive understanding of the complex ocean soundscape, shaped by both natural and anthropogenic sources, is essential.Ocean-bottom seismometer (OBS) datasets present an opportunity to explore the marine acoustic noise field. OBS hydrophones can be used to effectively study regional marine soundscapes. Additionally, the three-component OBS seismometers capture acoustic-to-seismic conversions from the water column, enabling the detection and localization of acoustic events.Through the analysis of OBS hydrophone recordings from the NE Atlantic offshore Ireland, we develop a frequency-dependent empirical sound propagation model for the area. The model provides information on (1) the spectral amplitude decay of sound sources in the region and (2) the distance between OBSs and acoustic sources.To construct the model, we consider ship noise from individual vessels recorded by OBSs at known distances from the source, using Automatic Identification System (AIS) ship-tracking data. As a ship moves away from its closest point of approach (CPA) to an OBS, the hydrophone records the evolution of the frequency-dependent ship's acoustic signal spectral amplitude with distance. This enables us to compute amplitude decay curves for narrow frequency bands from different vessel signals recorded at various OBSs. We apply a signal-to-noise ratio (SNR) quality control criterion and consider ships sailing along 'straight' paths at a constant speed (< 2 kn variation). After normalizing and scaling the CPAs of the different curves to a reference, we stack them for each frequency band and compute polynomial fits for the stacked amplitude decay curves. We analyse the number of vessel tracks needed to build a reliable empirical model. Numerical simulations are computed to further understand the acoustic propagation properties from our empirical model.The goal is to investigate (1) how spectral amplitudes at different frequencies behave with distance from the sound sources, (2) the efficiency of frequency-dependent sound propagation in the ocean in regions with variable seafloor morphologies, and (3) the potential use of the empirical model to obtain distances to other sound sources with unknown locations, such as Baleen whales. To achieve this, we compute ratios of spectral amplitudes for pairs of different frequency bands using the fitted amplitude decay curves. We test the model's ability to locate other vessels with known AIS positions before using it to locate different sound sources with unknown positions. Locating a sound source employing the distance provided by the empirical model is valuable when OBSs are too far apart for 'traditional' multi-sensor methods. In such cases, the incoming direction of the sound source is determined by rotating the horizontal-component OBS seismic data to point to the source back-azimuth. Also, the amplitude decay information provided by the model has multiple applications in other areas where noise pollution may be a concern (e.g., marine infrastructure construction).

Chapter 15 - A novel sound source localization method using a global-best guided cuckoo search algorithm for drone-based search and rescue operations

소리를 이용한 위치 추적은 마이크로폰을 이용하여 신호를 수집하고 수집된 신호로 부터 마이크로폰 간의 신호 도달 시간차를 추정한 뒤 추정된 시간차를 이용하여 소리의 발생 위치를 추정하는 과정을 거치게 된다. 실내 환경에서 이를 활용하기 위해서는 잡음과 반향음에 대한 강건성을 확보해야만 하는 제약이 따른다. 특히 실시간으로 구현하기 위해서는 계산의 효율성까지 고려되어야 한다. 본 논문에서는 네 개의 저가 콘덴서 마이크로폰을 이용하여 비용적인 측면과 계산량에서의 효율성을 모두 추구하였다. 네 개의 마이크로폰을 이용하여 마이크로폰 간의 소리 도달 시간차를 구하는 계산량을 줄였고 GCC-PHAT(Generalized Cross Correlation-Phase Transform) 알고리즘을 이용해서 강건성을 높였으며 iterative least square 방식을 이용하여 높은 정확도의 위치 데이터를 얻을 수 있었다. Sound localization algorithm usually adapts three step process: sampling sound signals, estimating time difference of arrival between microphones, estimate location of sound source. To apply this process in indoor environment, sound localization algorithm must be strong enough against reverberant and noisy condition. Additionally, calculation efficiency must be considered in implementing real-time sound localization system. To implement real-time robust sound localization system we adapt four low cost condenser microphones which reduce the cost and total calculation load. And to get TDOA(Time Differences of Arrival) of microphones we adapt GCC-PHAT(Generalized Cross Correlation-Phase Transform) which is robust algorithm to the reverberant and noise environment. The position of sound source was calculated by using iterative least square algorithm which produce highly accurate position data.

This study proposes a bi-directional recurrent neural network (Bi-RNN) post-processing method for speech enhancement (SE) at low signal-to noise ratios (SNR). Current speech enhancement solutions performed badly under low SNR situations. Loizou and Kim proposed a solution to reduce speech distortion errors in time-frequency (T-F) domain but it requires the knowledge of ground truth. As ground truth is unknown in real-life applications, the current study proposes to use a Bi-RNN to implement Loizou and Kim’s solution as a post-processing method for SE engines. Our solutions do not require prior knowledge of ground truth. The effectiveness of the proposed method is investigated with a spectral subtraction (SS) SE engine, a non-negative matrix factorization (NMF) SE engine, and a deep neural network ideal ratio mask (DNN-IRM) SE engine, under matched/mis-matched noise and different SNR conditions. Experimental results demonstrate that the proposed post-processing method effectively improved both perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) for all of these SE engines, especially at low SNR conditions.

With the rapid development of intelligent transportation, autonomous driving has become a core direction of technological innovation. Its safe and efficient operation relies on precise processing of multi-source sensor signals. Traditional signal processing methods, limited by manual feature design, often fail to extract complete features and suppress noise in complex traffic environments. This leads to perception biases and decision-making delays, posing safety hazards and restricting the implementation of autonomous driving technology. By combining literature review and technical analysis, this paper focuses on the integration of deep learning and autonomous driving signal processing, aiming to explore how to enhance the efficiency and reliability of signal processing through deep learning algorithms. It concludes that deep learning, through its adaptive feature learning ability, can significantly improve the real-time performance and accuracy of signal processing. However, challenges such as data scarcity, poor model interpretability, and high computational costs still exist. Future research should focus on technological breakthroughs such as lightweight model design and multi-sensor fusion to promote the large-scale application of deep learning in autonomous driving signal processing.