A new coding method for speech and audio signals

Speech signal is the primary mode of communication amongst humans. It is also used for the communication between human and machine, i.e., human-machine interaction (IoTs), communication with a range of electronic devices, e.g., Internet of Things (IoTs), and even with virtual assistants (VAs) or online search engines, e.g., Google etc. The use of speech signals for the communication over mobile phones, and smart phones based virtual assistants such as Cortana, Siri, Alexa etc. is well known. Usually, this communication involves some communication medium, which is susceptible to noise. The speech or audio signal used for the communication may be adversely affected by the noise that is added unwantedly over the communication channel or medium. It not only degrades the quality of communication and intelligibility of the message or content, but also leads to poor experience. The quality of speech signal may be degraded significantly, due to changes in the characteristics of speech signal, during the process from transmission to reception and the signal conversions involved at both ends. Hence there is a need for algorithms, signal processing methods, acoustic features and classifiers etc. for noise detection and noise reduction during the speech signal communication process. In this paper, a comprehensive review of various speech signal databases, speech signal processing methods used, acoustic features extracted and the classifiers used in different studies is presented. A comprehensive and comparative view of results obtained in different studies, and the applications is also presented. This study should be helpful to all the researchers working in the domains of speech signal processing in noisy environments or degraded speech etc.

Speech signal processing on graphs: The graph frequency analysis and an improved graph Wiener filtering method

The speech signal in general is corrupted by noise and the noise signal does not affect the speech signal uniformly over the entire spectrum. An improved Wiener filtering method is proposed in this paper for reducing background noise from speech signal in colored noise environments. In view of nonlinear variation of human ear sensibility in frequency spectrum, nonlinear multi-band Bark scale frequency spacing approach is used. The cross-correlation between the speech and noise signal is considered in the proposed method to reduce colored noise. To overcome harmonic distortion introduced in enhanced speech, in the proposed method regenerate the suppressed harmonics are regenerated. Objective and subjective tests were carried out to demonstrate improvement in the perceptual quality of speeches by the proposed technique.

Seismocardiogram (SCG) is a low-cost monitoring method to collect precordial vibrations of sternum due to heartbeats and evaluate cardiac activity. It is mostly used as an auxiliary measurement to the other monitoring methods; however, it carries significant patterns reflecting current cardiovascular health status of subjects. If it is properly collected within a non-clinical environment, it might be able to present preliminary data to physicians before clinic. SCG signals are morphologically noisy. These signals store excessive amount of data. Extracting significant information corresponding to heartbeat complexes is so important. Previously, the method called compressed sensing (CS) had been applied to weed up the redundant information by taking the advantage of sparsity feature in a study. This compressed sensing is based on storing significant signals below the Nyquist rate which suffice for medical diagnosis. It has been feasible to compress SCG signals with 3:1 compression rate at least while maintaining accurate signal reconstruction. Nevertheless, higher compression rates lead to the formation of artifacts on reconstructed signals. This limits a more aggressive compression to reduce the amount of data. The requirement of a different approach which will allow higher compression rates and lower loss of information arises. The purpose of this study is to obtain more competent results by using a method called predefined signature and envelope vector sets (PSEVS) which has been satisfyingly applied to electrocardiogram (ECG) and speech signals. In the study, simultaneously recorded ECG and SCG signals were modeled with the method called PSEVS. The reconstructed signals were compared to the original signals so as to investigate the efficacy of signature-based modeling methods in constructing medically remarkable biosignals for clinical use. After examining the components of reconstructed signals called frame-scaling coefficient, signature and envelope vectors, it has been seen that the error function values of envelope vectors differ from expected values. We concluded that reconstructed SCG signals were not adequate for medical diagnosis.

Speech can be expressed as a mechanism of expressing thoughts and ideas using vocal sounds. In humans, speech signals are generated at vocal cords, travelled through the vocal tract, and finally produced & transmitted through speaker’s mouth. These speech signals then usually travels through air or other mediums to the listener’s ear, where it acts as pressure waves. The bandwidth of speech signal is around 4 KHz. The noise produced by various ambient sources such as vehicles normally lies in this frequency range. Therefore, speech signals get easily distorted by the ambient noise. These distorted or degraded speech signals are called noisy speech signals. This paper focuses on speech processing (in particularly speech enhancement) of the noisy speech signals. This is very important as speech is the most commonly used way of communication and interaction between humans; however, it is very complex to understand. Therefore, this paper proposes an adaptive algorithm based on Wiener filter for speech enhancement. The proposed adaptive Wiener filter depends on the adaptation of the filter transfer function from sample to sample based on the speech signal statistics (mean and variance). The adaptive Wiener filter is implemented in time domain rather than in frequency domain. This is done to accommodate the random speech signal. The proposed method is compared to the traditional Wiener filter and the spectral subtraction methods.

Feature extraction from speech representation is one of the processes in speech recognition. Parametric modeling is a dominant approach to model speech signals. Within a localized interval, speech representation is equivalent to a noise driven output from an all-pole system that can be estimated using linear prediction. Besides the characteristics of speech, temporal variability of speech signal model is also due to the computation of linear prediction coefficients. Thus, an alternative representation is proposed based on the Gabor coefficients. In this paper, a comparison is made with the linear prediction coefficients to show the consistency of the parameters that are generated for implementation in the speech recognition system.

In this paper we propose a new approach based on energy-adaptive matching pursuits to improve sinusoidal modelling of speech and audio signals for coding and recognition purposes. To reduce the complexity of the algorithm, an over-complete dictionary composed of complex exponentials is used and an efficient implementation is presented. An analysis-synthesis windows scheme that avoids overlapping is proposed, too. Experimental results show evidence of the advantages of the proposed method for sinusoidal modelling of speech and audio signals compared to some others proposed in the literature.

In this paper, we consider the application of compressed sensing (aka compressive sampling) to speech and audio signals. We discuss the design considerations and issues that must be addressed in doing so, and we apply compressed sensing as a pre-processor to sparse decompositions of real speech and audio signals using dictionaries composed of windowed complex sinusoids. Our results demonstrate that the principles of compressed sensing can be applied to sparse decompositions of speech and audio signals and that it offers a significant reduction of the computational complexity, but also that such signals may pose a challenge due to their non-stationary and complex nature with varying levels of sparsity.

An ensemble 1D-CNN-LSTM-GRU model with data augmentation for speech emotion recognition

Auditory attention decoding (AAD) is the process of identifying the attended speech in a multi-talker environment using brain signals, typically recorded through electroencephalography (EEG). Over the past decade, AAD has undergone continuous development, driven by its promising application in neuro-steered hearing devices. Most AAD algorithms are relying on the increase in neural entrainment to the envelope of attended speech, as compared to unattended speech, typically using a two-step approach. First, the algorithm predicts representations of the attended speech signal envelopes; second, it identifies the attended speech by finding the highest correlation between the predictions and the representations of the actual speech signals. In this study, we proposed a novel end-to-end neural network architecture, named AADNet, which combines these two stages into a direct approach to address the AAD problem. We compare the proposed network against traditional stimulus decoding-based approaches, including linear stimulus reconstruction, canonical correlation analysis, and an alternative non-linear stimulus reconstruction using three different datasets. AADNet shows a significant performance improvement for both subject-specific and subject-independent models. Notably, the average subject-independent classification accuracies for different analysis window lengths range from 56.3% (1 s) to 78.1% (20 s), 57.5% (1 s) to 89.4% (40 s), and 56.0% (1 s) to 82.6% (40 s) for three validated datasets, respectively, showing a significantly improved ability to generalize to data from unseen subjects. These results highlight the potential of deep learning models for advancing AAD, with promising implications for future hearing aids, assistive devices, and clinical assessments.

Currently, the use of speech recognition is increaseingly in many applications such as mobile device interaction, interactive voice response system, voice search, voice dictation and voice identification. The heart of such applications is speech features needed to represent input signals. However, in real applications, speech signals are sampled with various sampling rates. The different sampling rates of input speech lead to the different features. This makes the speech recognition rate dropping. Therefore, this paper proposes an independent resolution descriptor based on fractal codes obtained by fractal encoding and decoding processes. The encoding process extracts fractal codes from partitioned speech signals, whereas the decoding process reconstructs independent resolution speech signals from the fractal codes. This method can effectively reconstruct speech signals at any sampling rates, especially at a higher sampling rate, which is a grand challenge. The proposed method is evaluated the performance by testing with AN4 corpus of CMU Sphinx speech recognition engine. The experimental results show that the proposed method can improve the accuracy of speech recognition, even if the sampling rate of testing speeches differs from that of training speeches.

Compressed Sensing (CS) is an emerging theory which can sample the sparse signal or compressible signal via sub-Nyquist sampling rate and reconstruct the original signal with small amount of measurements. Since speech signal is sparse in Discrete Cosine Transform (DCT) domain, a kind of adaptive Bayesian Compressed Sensing (BCS) based on speech signal is proposed in this paper. In the one hand, our proposed method exploits the difference of energy within different speech frame to allot measurements adaptively for each speech frame aim to promote the quality of recovery speech signal. In the other hand, the position information of sparse coefficient in each speech frame is also utilized by our proposed method to recover its neighboring speech frame for reducing the recovery time of speech signal. The experimental results prove that our proposed method is surely effective and practical.

A parametric loudspeaker has been used for audio guidance to a specific area because it has a sharper directivity compared with the conventional electrodynamic loudspeakers. The parametric loudspeaker emits an ultrasound as a carrier wave, which is modulated with an audio signal and has large-amplitude. An audible sound is reproduced by the modulated ultrasound with large-amplitude distorted by the nonlinearity on the air. The conventional modulations have been proposed as the amplitude modulation and the frequency modulation. In the sound quality, the amplitude modulation is superior to the frequency modulation. However, in the sound pressure level, the frequency modulation is superior to the amplitude modulation. In the present paper, we especially focus on that the parametric loudspeaker will emit the speech signals for the audio guidance. Therefore, we propose new modulation method based on the combination of amplitude and frequency modulations, which are specialized for speech signals. More specifically, we apply the amplitude and frequency modulations to the divided frequency bands of speech signals, respectively. In order to confirm the effectiveness of the proposed method, we carried out evaluation experiments. As a result of evaluation experiments, we confirmed the effectiveness of the proposed method compared with conventional methods.

Pitch estimation is an essential task in audio processing due to its key role in many speech and music applications. Still, accurately predicting a continuous value from a high range of pitch frequencies is a challenging task. Inspired by the success of signal processing filterbank methods, we propose a novel deep architecture for accurate pitch estimation. The proposed method is composed of an encoder and multiple decoders. The encoder is implemented by a convolutional neural network that provides a good representation of the raw audio signal, and its output is fed into a set of decoders. Each decoder predicts the pitch value within a specific frequency band and is implemented by a fully-connected neural network. Such a construction allows each decoder to specialize in a particular frequency regime, which turns into a more accurate estimation of pitch values for music and speech signals.

In this paper, we propose a method for decomposing speech signals, evaluating the discriminative, and determining the representative vectors of signal sets. At first, we decompose all speech signals with the level four Db4 wavelet decomposition to reconstruct the approximation sub-signals of all four levels, transfer all the speech signals and the sub-signals into the Linear Prediction Codes (LPC), and calculate the Difference vectors between the LPC (DLPC) of the speech signals and that of the approximation sub-signals of different levels. We adopt those LPC and DLPC vectors as the feature vectors, evaluate the discriminative of each set of feature vectors, and adopt the mean vectors of the clusters as the representative vectors. We can use those to set the parameters of the speech recognizer. Although the experimental results are only valid for the speech signals and the wavelet functions used for this experimental study, it should provide valuable references in general applications.