A multichannel beamforming-based framework for speech extraction

Beamforming techniques are becoming popular in a wide spectrum of applications thanks to their promising performance. We consider applying the minimum variance distortionless response (MVDR) beamformer in a dynamic scenario, where the desired speaker, the noise components or microphone array may move. The generalized sidelobe canceler (GSC) structure offers a convenient implementation of the MVDR, allowing to track the noise efficiently. Yet, noise reduction is degraded during speech tracking. We propose a QR Decomposition (QRD) based MVDR beamformer, which allows to track both speech and noise variations in dynamic scenarios without sacrificing noise reduction during speech segments. The proposed method utilizes the QRD algorithm for the implementation of a complete system required for the realization of an MVDR beamformer, which includes the identification of noisy speech or noise-only time-frequency bins, speech tracking and noise tracking. The proposed QRD based MVDR is efficiently implemented, and its overall computational complexity is O(M2), with M being the number of microphones. The advantages of the proposed method over the GSC are exemplified in a synthetically simulated scenario.

This paper studies and discusses robust minimum variance distortionless response (MVDR) beamformer with subarray multistage processing for circular hydrophone arrays, which is an extension of the multistage MVDR beamformer with line microphone arrays for noise reduction. The basic theory is to divide the hydrophones into a certain number of groups with each group only having two hydrophones for MVDR beamforming. After the beamforming, it is equivalent that the number of hydrophones is half of that at the adjoining former stage. Then the grouping and beamfoming processing will be repeated till that there is only one hydrophone. This subarray multistage MVDR beamformer has the following properties in the isotropic noise field: first, it is much more robust than the conventional MVDR beamformer; second, its array gain is lower than that of the conventional MVDR beamformer; moreover, the larger the sensor number in each subarray is, the higher the array gain will be, but with lower white noise gain. By choosing a proper hydrophone number of each subarray, a tradeoff can be made between the array gain and robustness.

Microphone arrays based on the minimum variance distortionless response (MVDR) beamformer are among the most popular for speech enhancement applications. The original MVDR is excessively sensitive to source location and microphone gains. Previous research has made MVDR practical by successfully increasing the robustness of MVDR to source location, and MVDR-based microphone arrays are already commercially available. Nevertheless, MVDR performance is still weak in cases where microphone gain variations are too large, e.g., for circular arrays of directional microphones. In this paper we propose an improved MVDR beamformer which takes into account the effect of sensors (e.g. microphones) with arbitrary, potentially directional responses. Specifically, we form estimates of the relative magnitude responses of the sensors based on the data received at the array and include those in the original formulation of the MVDR beamforming problem. Experimental results on real-world audio data show an average 2.4 dB improvement over conventional MVDR beamforming, which does not account for the magnitude responses of the sensors.

Submarine hydrophone arrays sample the underwater acoustic pressure field in space and time to sense the presence of sources of radiated sound and to extract tactical information from the received sounds. The outputs of the hydrophones are combined by a spatial filter (or beamformer) so that signals from a chosen direction are coherently added while the effects of noise and interference from other directions are reduced by destructive interference. The spatial filter appropriately weights the hydrophone outputs prior to summation so as to enhance the output signal-to-noise ratio, thereby improving the detection, classification, localization and tracking performance of the passive sonar system onboard the submarine. By processing real acoustic data from a submarine hull-mounted array, it is shown that an optimal (adaptive) frequency-domain spatial filter (based on inversion of the observed cross spectral matrix for each frequency bin) enhances the detection of underwater acoustic signals, when compared with conventional spatial filtering (or delay-and-sum beamforming). The degree of improvement, however, depends on the method used to normalize the cross spectral matrix. Superior detection performance for a submarine hull-mounted array occurs when the observed cross spectral matrix for each frequency of interest is normalized using the average of all the single hydrophone output powers at each frequency. It is also shown that the self-noise analysis of a submarine towed array is facilitated by processing the element-level data in the frequency-wave number domain with a constrained optimal spatial filter, which is also referred to as a Minimum Variance Distortionless Response (MVDR) beamformer. This spatiotemporal filtering method is employed to separate signals (surface ship contacts) and tow vessel noise components (direct path and multipath arrivals) according to their directions of propagation. This method is also found to separate spatially-correlated self-noise components that propagate within the towed array structure at a speed that is slower than the speed of sound travel in water. A comparison of the frequency-wave number power spectra estimated using conventional and optimal spatial filtering methods shows that the MVDR spatial filter enables the various sources of acoustic energy that are sensed by the array to be more clearly delineated in frequency-wave number space. The MVDR spatial filter is a data-adaptive spatial filter that is observed to suppress spatial leakage, to enhance the spatial resolution of a towed array through narrower beamwidths, and to provide superdirective array gain at frequencies well below the design frequency of the towed array. Frequency-wave number analysis with optimal spatial filtering is a powerful diagnostic tool for studying the self-noise characteristics of a towed linear array of hydrophones.

The "standard" minimum variance distortionless response (MVDR) beamforming algorithm of McWhirter and Shepherd (1989) is known to suffer from linear roundoff error buildup. An alternative has been developed by Moonen (1993) based on the so-called inverse QR-updating algorithm. It is amenable to pipelined implementation, but unlike the McWhirter and Shepherd algorithm, it is stable numerically. Here, a variant of the latter algorithm that combines (part of) the McWhirter and Shepherd procedure into the inverse QR updating scheme is discussed. This makes the derivation of the algorithm less complex compared to the original. Furthermore, by making use of a residual extraction property of the inverse updating procedure, an interesting correspondence of the new MVDR algorithm with so-called "generalized sidelobe cancellation" is explicitly revealed.

Background and Objectives: One major problem in the minimum power distortionless response (MPDR) beamformer is the signal cancellation problem, i.e., the desired signal is canceled by the reflected signal, even though the distortionless response constraint is satisfied. Solving this problem is the objective of this paper. Methods: It is well known that the signal cancellation problem can be avoided by minimizing the cross-spectrum matrix of noise, i.e., using the minimum variance distortionless response (MVDR) beamformer. But, in the case of disturbance signals which have correlation with the desired signal, estimation of this matrix is a challenging problem. In this paper we propose an approach for estimating the cross-spectrum matrix of noise signal from which we can solve the signal cancellation problem. Results: Simulation examples show that using the proposed method we can bypass the signal cancellation problem completely. Conclusion: A common belief is that in the case of a disturbance that is a reflected version of the desired signal, due to cohesive appearance and disappearance of both the disturbance and the desired signal, the estimation of cross-spectrum matrix of noise signal is typically not possible in practice. So, based on this common belief, we can’t use the MVDR beamformer in this case. In this paper, we show that this common belief is a fault. We propose a general approach for estimating the cross-spectrum matrix of noise signal that is applicable even in the case of correlated disturbances.

Microphone array (MA) own substantial gain in evaluation due to the high spatial diversity to suppress unwanted noise, third-party talker, background noise, interfering signals coming from undesired directions. Minimum Variance Distortionless Response (MVDR) beamformer is an effective solution for extracting the clean speech data while mitigating all total noise power in different recording environment. However, in realistic situation, because of the moving head, the error of estimating transfer function, the complex and annoying recording scenario, the different microphone sensitivities, the inaccurate MA configuration, the imprecise estimation of other acoustic factors, MVDR beamformer’s evaluation often decreased in the the term of speech quality and speech intelligibility. The existence of remained noise, speech distortion decrease the overall MVDR beamformer’s performance. In this paper the author proposed using an appropriate spectral mask for reducing speech distortion. The obtained promising result has confirmed the effectiveness of the author’s method for improving the MVDR beamformer’s performance in adverse recording situations.

It remains challenging for robots to accurately perform sound source localization and speech recognition in a real environment with reverberation, noise, and the voices of multiple speakers. Accordingly, we propose “U-TasNet-Beam,” a speech extraction method for extracting only the target speaker’s voice from all ambient sounds in a real environment. U-TasNet-Beam is a neural beamformer comprising three elements: a neural network for removing reverberation and noise, a second neural network for separating the voices of multiple speakers, and a minimum variance distortionless response (MVDR) beamformer. Experiments with simulated data and recorded data show that the proposed U-TasNet-Beam can improve the accuracy of sound source localization and speech recognition in robots compared to the conventional methods in a noisy, reverberant, and multi-speaker environment. In addition, we propose the spatial correlation matrix loss (SCM loss) as a loss function for the neural network learning the spatial information of the sound. By using the SCM loss, we can improve the speech extraction performance of the neural beamformer.

In this paper, a multi-channel speech enhancement method with the minimum variance distortionless response (MVDR) beamforming method based on the time-frequency (T-F) masking is proposed. In this study, First, the logarithmic power spectrum (LPS) features of multichannel signals are used as input features to estimate a T-F mask of the reference microphone by the deep neural network (DNN) model. Then, the estimated mask is utilized to calculate speech covariance matrix that is used to estimate a steering vector for constructing the MVDR beamformer. The steering vector is estimated by the generalized eigenvalue decomposition (GEVD) method. Finally, the output speech of the beamformer is processed by the DNN-based IRM model. In order to prove the effectiveness of the proposed method, the perceptual evaluation of speech quality (PESQ) and the segment signal-to-noise ratio (SSNR) are employed. The experimental results show that the proposed method effectively increased the PESQ and SSNR.

The optimal microphone array includes two processing blocks ‐ minimum variance distortionless response (MVDR) beamformer and the single‐channel Wiener filter, which acts as post‐filter. The main drawback of MVDR beamformer is the cancellation of the desired speech signal and its degradation in multi‐path wave propagation environment. To make the adaptive algorithm robust against room reverberation and to prevent desired signal cancellation, an estimation of the unknown desired speaker's transfer function was proposed. The estimation is based on the imperfect signal and the interference covariance matrices estimated from available microphone signals during speaker activity and pause of speech respectively. As MVDR beamformer suppresses coherent interference, post‐filter has to reduce diffuse acoustic noise. The post‐filter proposed in this paper is developed under assumption that complex coherence function is unknown but time invariant. The additional improvement of the post‐processing algorithm on low frequencies is obtained by combining a priori noise power attenuation factor for diffuse noise field with estimated one. The proposed algorithm is tested on simulated room with reverberation, and compared with some known post‐processing algorithms with rather good results.

The minimum variance distortionless response (MVDR) beamformer is very sensitive to errors, such as the the finite snapshots and the steering vector errors. A small mismatch can lead to serious degradation to its performance, especially at high signal-to-noise ratio(SNR). In this paper, a new robust MVDR beamformer is proposed. Our analysis starts from the physical meaning of the MVDR weight vector formula. Based on this analysis, DOA matrix is constructed in the single uniform linear array. Through the decomposition of this DOA matrix, we could obtain the actual signal modes contained in the covariance matrix of the snapshots. Then the weight vector could be modified to solve the robustness problem of MVDR beamforming. The computer simulation shows the effectiveness of this method.

Acoustic signals received by platform mounted sonar arrays can be spatially processed to enhance the detection of targets in the presence of both ambient and platform generated (self) noise. Ambient noise in the ocean, such as that due to distant shipping or biological choruses, are known to be spatially correlated. The platform generated noise will be of near-field origin and may not be received by all elements in the array. In this paper we investigate the performance of the minimum variance distortionless response (MVDR) beamformer and the recently introduced Fourier integral method (FIM) and compare their performances with the conventional beamformer. Real passive sonar data, obtained from a platform mounted sparse linear array of hydrophones, is used to study the performance of the beamformers in a typical sonar environment. It is shown that in the absence of self noise, when the array is accurately calibrated the MVDR beamformer will perform very well, but when sensor gain or phase errors are present the performance of the MVDR beamformer is degraded. Further, the MVDR beamformer is unable to reject the self noise which is not "seen" by the entire array. FIM however seems to perform well and a modified version of FIM, which we call weighted FIM (WFIM), is shown to perform better and is at worst comparable to a well calibrated MVDR beamformer.

A new approach to implement minimum variance distortionless response (MVDR) beamforming is introduced for coherent plane-wave compounding (CPWC). MVDR requires the covariance matrix of the incoming signal to be estimated and a spatial smoothing approximation is usually adopted to prevent this calculation from being underconstrained. In the new approach, we analyze MVDR as a spatial filter that decorrelates signals received at individual channels before summation. Based on the analysis, we develop two MVDR beamformers without using any spatial smoothing. First, MVDR weights are applied to the received signals after accumulating the data over transmits at different angles, while the second involves weighting the data collected in individual transmits and compounding over the transducer elements. In both cases, the covariance matrix is estimated using a set of slightly different combinations of the echo data. We show the sufficient statistic for this estimation that can be described by approximating the correlation among the backscattered ultrasound signals to their spatial coherence. Using the van Cittert-Zernike theorem, their statistical similarity is assessed by relating the spatial coherence to the profile of the source intensity. Both spatial-coherence-based MVDR beamformers are evaluated on data sets acquired from simulation, phantom, and in vivo studies. Imaging results show that they offer improvements over simple coherent compounding in terms of spatial and contrast resolutions. They also outperform other existing MVDR-based methods in the literature that are applied to CPWC.

This paper presents the minimum variance distortionless response (MVDR) beamformer combined with a Speech Enhancement (SE) gain function as a real-time application running on smartphones that work as an assistive device to Hearing Aids. It has been shown that beamforming techniques improve the Signal to Noise Ratio (SNR) in noisy conditions. In the proposed algorithm, MVDR beamformer is used as an SNR booster for the SE method. The proposed SE gain is based on the Log-Spectral Amplitude estimator to improve the speech quality in the presence of different background noises. Objective evaluation and intelligibility measures support the theoretical analysis and show significant improvements of the proposed method in comparison with existing methods. Subjective test results show the effectiveness of the application in real-world noisy conditions at SNR levels of -5 dB, 0 dB, and 5 dB.

Purely neural network (NN) based speech separation and enhancement methods, although can achieve good objective scores, inevitably cause nonlinear speech distortions that are harmful for the automatic speech recognition (ASR). On the other hand, the minimum variance distortionless response (MVDR) beamformer with NN-predicted masks, although can significantly reduce speech distortions, has limited noise reduction capability. In this paper, we propose a multi-tap MVDR beamformer with complex-valued masks for speech separation and enhancement. Compared to the state-of-the-art NN-mask based MVDR beamformer, the multi-tap MVDR beamformer exploits the inter-frame correlation in addition to the inter-microphone correlation that is already utilized in prior arts. Further improvements include the replacement of the real-valued masks with the complex-valued masks and the joint training of the complex-mask NN. The evaluation on our multi-modal multi-channel target speech separation and enhancement platform demonstrates that our proposed multi-tap MVDR beamformer improves both the ASR accuracy and the perceptual speech quality against prior arts.