Dialect Identification in Telugu Language Speech Utterance Using Modified Features with Deep Neural Network

Chaha is a very low-resource language, which is suffered from lack of language resources to develop human language technologies, namely, speech recognition. Moreover, Chaha writing system is syllabic with a consonant-vowel (CV) syllable structure. The Chaha orthography is a one-to-one correspondence with syllable sound units. By considering the above facts of Chaha, this study is the first endeavor that explores the use of CV syllables as acoustic modeling units for developing speech recognizers, using the Gaussian mixture model (GMM) and unilingual and transfer learning deep neural network (DNN) models. Our experimental results demonstrate that the syllablebased unilingual DNN and transfer learning DNN models outperform the corresponding GMM and unilingual DNN models with absolute performance improvements of 2.8 to 3.09% and 1.07 to 4.94%, respectively. The best performing syllable-based recognizer is achieved using a shared hidden layer (SHL) time delay deep neural network (TDNN) model with a word error rate (WER) of 23.11%. Hence, the CV syllables are suitable acoustic units to develop Chaha speech recognition systems under sufficient training corpus.

Speaker identification refers to the process of recognizing human voice using artificial intelligence techniques. Speaker identification technologies are widely applied in voice authentication, security and surveillance, electronic voice eavesdropping, and identity verification. In the speaker identification process, extracting discriminative and salient features from speaker utterances is an important task to accurately identify speakers. Various features for speaker identification have been recently proposed by researchers. Most studies on speaker identification have utilized short-time features, such as perceptual linear predictive (PLP) coefficients and Mel frequency cepstral coefficients (MFCC), due to their capability to capture the repetitive nature and efficiency of signals. Various studies have shown the effectiveness of MFCC features in correctly identifying speakers. However, the performances of these features degrade on complex speech datasets, and therefore, these features fail to accurately identify speaker characteristics. To address this problem, this study proposes a novel fusion of MFCC and time-based features (MFCCT), which combines the effectiveness of MFCC and time-domain features to improve the accuracy of text-independent speaker identification (SI) systems. The extracted MFCCT features were fed as input to a deep neural network (DNN) to construct the speaker identification model. Results showed that the proposed MFCCT features coupled with DNN outperformed existing baseline MFCC and time-domain features on the LibriSpeech dataset. In addition, DNN obtained better classification results compared with five machine learning algorithms that were recently utilized in speaker recognition. Moreover, this study evaluated the effectiveness of one-level and two-level classification methods for speaker identification. The experimental results showed that two-level classification presented better results than one-level classification. The proposed features and classification model for identifying a speaker can be widely applied to different types of speaker datasets.

Speech production knowledge has been used to enhance the phonetic representation and the performance of automatic speech recognition (ASR) systems successfully. Representations of speech production make simple explanations for many phenomena observed in speech. These phenomena can not be easily analyzed from either acoustic signal or phonetic transcription alone. One of the most important aspects of speech production knowledge is the use of articulatory knowledge, which describes the smooth and continuous movements in the vocal tract. In this paper, we present a new articulatory model to provide available information for rescoring the speech recognition lattice hypothesis. The articulatory model consists of a feature front-end, which computes a voicing feature based on a spectral harmonics correlation (SHC) function, and a back-end based on the combination of deep neural networks (DNNs) and hidden Markov models (HMMs). The voicing features are incorporated with standard Mel frequency cepstral coefficients (MFCCs) using heteroscedastic linear discriminant analysis (HLDA) to compensate the speech recognition accuracy rates. Moreover, the advantages of two different models are taken into account by the algorithm, which retains deep learning properties of DNNs, while modeling the articulatory context powerfully through HMMs. Mandarin speech recognition experiments show the proposed method achieves significant improvements in speech recognition performance over the system using MFCCs alone.

Significance of analytic phase of speech signals in speaker verification

In this paper, we propose a feature set by concatenating Mel-Frequency Cepstral Coefficients (MFCC) and Spectral Centroid Magnitude Coefficients (SCMC) features for Acoustic Scene Classification (ASC) using Deep Neural Networks (DNN). MFCC features are used to hold the acoustic characteristics such as spectral envelope of an acoustic scene in each frame. It also carries the sub-band average energy as a single dimension. SCMC features are used to hold the distribution of energy in a sub-band effectively. A test is carried out on Tampere University of Technology (TUT) Acoustic Scenes 2017 Dataset. The DNN architecture for utterance level classification has been used. The proposed system’s performance on a 4-fold cross-validation setup is 80.2% and it gives 5.4% relative improvement in performance when compared to the baseline system that uses log-Mel band energies with Multi-Layer Perceptron model.

Interconnected devices are becoming attractive solutions to integrate physical parameters and making them more accessible for further analysis. Edge devices, located at the end of the physical world, measure and transfer data to the remote server using either wired or wireless communication. The exploding number of sensors, being used in the Internet of Things (IoT), medical fields, or industry, are demanding huge bandwidth and computational capabilities in the cloud, to be processed by Artificial Neural Networks (ANNs) – especially, processing audio, video and images from hundreds of edge devices. Additionally, continuous transmission of information to the remote server not only hampers privacy but also increases latency and takes more power. Deep Neural Network (DNN) is proving to be very effective for cognitive tasks, such as speech recognition, object detection, etc., and attracting researchers to apply it in edge devices. Microcontrollers and single-board computers are the most commonly used types of edge devices. These have gone through significant advancements over the years and capable of performing more sophisticated computations, making it a reasonable choice to implement DNN. In this thesis, a DNN model is trained and implemented for Keyword Spotting (KWS) on two types of edge devices: a bare-metal embedded device (microcontroller) and a robot car. The unnecessary components and noise of audio samples are removed, and speech features are extracted using Mel-Frequency Cepstral Co-efficient (MFCC). In the bare-metal microcontroller platform, these features are efficiently extracted using Digital Signal Processing (DSP) library, which makes the calculation much faster. A Depth wise Separable Convolutional Neural Network (DSCNN) based model is proposed and trained with an accuracy of about 91% with only 721 thousand trainable parameters. After implementing the DNN on the microcontroller, the converted model takes only 11.52 Kbyte (2.16%) RAM and 169.63 Kbyte (8.48%) Flash of the test device. It needs to perform 287,673 Multiply-and-Accumulate (MACC) operations and takes about 7ms to execute the model. This trained model is also implemented on the robot car, Jetbot, and designed a voice-controlled robotic vehicle. This robot accepts few selected voice commands-such as “go”, “stop”, etc. and executes accordingly with reasonable accuracy. The Jetbot takes about 15ms to execute the KWS. Thus, this study demonstrates the implementation of Neural Network based KWS on two different types of edge devices: a bare-metal embedded device without any Operating System (OS) and a robot car running on embedded Linux OS. It also shows the feasibility of bare-metal offline KWS implementation for autonomous systems, particularly autonomous vehicles.

Over the past few decades, research has focused heavily on automatic speech recognition (ASR). Although ASR for a few languages is close to reality; ASR for the low resource languages like Malayalam is still in its infancy. Here, in this work, the authors discuss the experiment conducted on accented data of Malayalam speech using two approaches. One approach uses the spectral features for modeling using deep convolutional neural network and the other uses the influential features of speech signals for modeling using LSTM-RNN approach. The proposed methodology comprises three distinct stages; dataset preparation, feature extraction, classification, and hence the construction of deep learning models that recognize the accent-based spoken sentences in the Malayalam language. Mel-frequency cepstral coefficient (MFCC) algorithm, short-term Fourier transform (STFT), and mel spectrogram methodologies are used for feature engineering and hence the features that represent the speech signals are used for constructing the accented ASR system for the Malayalam language using long short-term memory (LSTM) a recurrent neural network (RNN). The spectrogram dataset has been constructed for the speech dataset and used for constructing the ASR model with deep convolutional neural network (DCNN). The result of the experiment shows that LSTM-based RNN outperforms DCNN for the proposed dataset that has been constructed for the experiment in the natural recording environment.

The aim of this paper is to exhibit a comparative case study of the conventional speech recognition GMM-HMM (Gaussian mixture model - hidden Markov model) architecture and the recent model based on deep neural networks. During years the GMM approach has controlled the speech recognition tasks, however it has been surpassed with the resurgence of artificial neural networks. To exemplify these acoustic modeling frameworks, a case study has been conducted by using the Kaldi toolkit, employing a personalized speaker-independent mid-vocabulary voice corpus for recognition of digit strings and personal name lists in latin spanish on a connected-words phone dialing task. The speech recognition accuracy obtained in the results shows a better word error rate by using the DNN acoustic modeling. A 20.71% relative improvement is obtained with DNN-HMM models (3.33% WER) in respect to the lowest GMM-HMM rate (4.20% WER).

Objective: Accurate dialect identification technique helps in improving the speech recognition systems that exist in most of the present day electronic devices and is also expected to help in providing new services in the field of e-health and telemedicine which is especially important for older and homebound people. Methods: In this paper we have developed the speech corpora needed for the dialect identification purpose and described a method to identify Assamese language (an Indian language) and two of its dialects, viz., Kamrupi and Goalparia. Findings: Research work done on dialect identification is relatively much less than that on language identification for which one of the reasons being dearth of sufficient database on dialects. As mentioned, we have developed the database and then Mel-Frequency Cepstral Coefficient has been used to extract the spectral features of the collected speech data. Two modeling techniques, namely, Gaussian Mixture Model and Gaussian Mixture Model with Universal Background Model have been used as the modeling techniques to identify the dialects and language. Novelty: So far, standard speech database for Assamese dialects that can be used for speech processing research has not been made. In this paper, we not only describe a method to identify dialects of the Assamese language, but have also developed the speech corpora needed for the dialect identification purpose. Keywords: Assamese, Gaussian Mixture Model, Gaussian Mixture Model with Universal Background Model, Goalparia, Kamrupi, Mel-Frequency Cepstral Coefficient

From many years there has been gaining interest in the field of SER using Matlab. SER states the emotional state by analyzing the input speech. SER has a simple pattern and also including feature extraction, feature matching, classification and database. Here, from the input speech by using algorithm features are extracted by using some models the feature matching takes place. By this process we use to analyze the characteristics of the input speech signal. Hence, the system recognize the state of emotion. The system states some the of emotions: Angry, Boredom, Anxiety, Disgust,Happiness, Neutral, Sadness. The main purpose of this paper is to give survey on two of the algorithms using HMM model with different speech emotion databases. There are several audio features for extracting are available. And also various classifiers are available. The most popular models are Hidden Markov model(HMM), Vector Quantization(VQ), Gaussian Mixture model(GMM), Deep Neural Networks (DNN), Neural Networks(NN) and Artificial Neural Networks(ANN). There are several speech emotion databases included Berlin database. Hence, we reviewed some of the models will be discussed in this paper.

The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

Automated image classification is an essential task of the computer vision field. The tagging of images into a set of predefined groups is referred to as image classification. The implementation of computer vision to automate image classification would be beneficial because manual image evaluation and identification can be time-consuming, particularly when there are many images of different classes. Deep learning approaches are proven to overperform existing machine learning techniques in many fields in recent years, and computer vision is one of the most notable examples. The very deep neural network (VDCNN) is a powerful deep learning model for image classification, and this paper examines it briefly using MNIST handwritten digit dataset. This dataset is used to prove the efficacy of very deep neural networks over other deep learning models. The proposed study aims to comprehend the very deep neural network architecture used to accomplish a handwritten digit recognition task. The feasibility of the proposed model is evaluated using mean accuracy, validation accuracy, and standard deviation. The study results of the very deep neural network model are compared to a convolutional neural network and convolutional neural network with batch normalization. According to the results of the comparison study, very deep neural networks achieve high accuracy of 99.1% for a handwritten dataset. The outcome of the proposed work is used to interpret how well a very deep neural network performs when compared to the other two models of deep neural networks. This proposed architecture may be used to automate the classification of handwritten digits dataset.

In recent years, with the rapid development of Deep Learning (DL) and the widespread uses of Deep Neural Networks (DNN), speech recognition technology has attracted great attention. This technology enables machines to convert human speech into text and plays an important role in some areas such as virtual assistants, conference transcription and smart home devices. Traditional models such as Hidden Markov Models (HMM) and Gaussian Mixture Models (GMM) used to play a key role in modeling and recognizing speech signals. Even though these methods still have challenges in dealing with high‐dimensional feature vectors as they struggle to efficiently learn large amounts of model parameters while maintaining good generalization performance. The emergence of DNNs has revolutionized the field of speech recognition, making it possible to automatically extract features from raw speech signals. Thereby, modern speech recognition systems can exhibit higher adaptability and generalization capabilities, achieving higher accuracy and stability under various conditions. This review explores the application of DL techniques to speech recognition, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and Transformer‐based architectures, etc. Advances in these techniques have dramatically improved recognition accuracy, noise immunity, and adaptability to multiple accents, thereby improving the total efficiency of automatic speech systems. By leveraging advanced feature extraction and modeling capabilities, deep learning‐based systems support real‐ time processing and minimize the dependency on traditional manually designed features. However, there are still limitations in terms of interpretability, computational efficiency, and fairness to different accents and languages, which impede the widespread use of these techniques. Future research should focus on solving these issues in order to achieve fairer, efficient, and scalable deep learning‐driven speech recognition schemes.

Multi-microphone speech recognition integrating beamforming, robust feature extraction, and advanced DNN/RNN backend

This paper extends an earlier work on designing a speech recognition system based on Hidden Markov Model (HMM) classification technique of using visual modality in addition to audio modality[1]. Improved off traditional HMM-based Automatic Speech Recognition (ASR) accuracy is achieved by implementing a technique using either RNN-based or CNN-based approach. This research is intending to deliver two contributions: The first contribution is the methodology of choosing the visual features by comparing different visual features extraction methods like Discrete Cosine Transform (DCT), blocked DCT, and Histograms of Oriented Gradients with Local Binary Patterns (HOG+LBP), and applying different dimension reduction techniques like Principal Component Analysis (PCA), auto-encoder, Linear Discriminant Analysis (LDA), t-distributed Stochastic Neighbor Embedding (t-SNE) to find the most effective features vector size. Then the obtained visual features are early integrated with the audio features obtained by using Mel Frequency Cepstral Coefficients (MFCCs) and feed the combined audio-visual feature vector to the classification process. The second contribution of this research is the methodology of developing the classification process using deep learning by comparing different Deep Neural Network (DNN) architectures like Bidirectional Long-Short Term Memory (BiLSTM) and Convolution Neural Network (CNN) with the traditional HMM. The proposed model is evaluated on two multi-speakers AV-ASR datasets named AVletters and GRID with different SNR. The model performs speaker-independent experiments in AVlettter dataset and speaker-dependent in GRID dataset.