Security-Monitoring using Microphone Arrays and Audio Classification

Content-based audio classification and segmentation is a basis for further audio/video analysis. In this paper, we present our work on audio segmentation and classification which employs support vector machines (SVMs). Five audio classes are considered in this paper: silence, music, background sound, pure speech, and non- pure speech which includes speech over music and speech over noise. A sound stream is segmented by classifying each sub-segment into one of these five classes. We have evaluated the performance of SVM on different audio type-pairs classification with testing unit of different- length and compared the performance of SVM, K-Nearest Neighbor (KNN), and Gaussian Mixture Model (GMM). We also evaluated the effectiveness of some new proposed features. Experiments on a database composed of about 4- hour audio data show that the proposed classifier is very efficient on audio classification and segmentation. It also shows the accuracy of the SVM-based method is much better than the method based on KNN and GMM.

In a cloud setting where audio is intercepted from multiple nodes, it is of interest of identify the type of audio present at each node. Each node can be seen as an IOT (Internet of Things) device. Audio type can be speech by a particular speaker, different kinds of music or background noises, whose monitoring is useful for various IOT applications. A supervised audio classification system using sparse representation over a cloud network is presented. In this system, dictionaries are learnt from different audio streams on multiple nodes in the training stage. Audio classification is done separately on different nodes using distributed sparse representation, avoiding any centralized processing. Both training and testing is done in a distributed manner, and the final estimate of the audio type is arrived by consensus among the nodes. Given an audio segment at any node, distributed sparse coding is used to classify the segment into one of the audio classes using the different dictionary models learnt at different nodes.

We focus the attention on the area of generic and automatic audio classification and segmentation for audio-based multimedia indexing and retrieval applications. In particular, we present a fuzzy approach toward hierarchic audio classification and global segmentation framework based on automatic audio analysis providing robust, bi-modal, efficient and parameter invariant classification over global audio segments. The input audio is split into segments, which are classified as speech, music, fuzzy or silent. The proposed method minimizes critical errors of misclassification by fuzzy region modeling, thus increasing the efficiency of both pure and fuzzy classification. The experimental results show that the critical errors are minimized and the proposed framework significantly increases the efficiency and the accuracy of audio-based retrieval especially in large multimedia databases.

An on-line audio classification and segmentation system is presented in this research, where audio recordings are classified and segmented into speech, music, several types of environmental sounds and silence based on audio content analysis. This is the first step of our continuing work towards a general content-based audio classification and retrieval system. The extracted audio features include temporal curves of the energy function,the average zero- crossing rate, the fundamental frequency of audio signals, as well as statistical and morphological features of these curves. The classification result is achieved through a threshold-based heuristic procedure. The audio database that we have built, details of feature extraction, classification and segmentation procedures, and experimental results are described. It is shown that, with the proposed new system, audio recordings can be automatically segmented and classified into basic types in real time with an accuracy of over 90 percent. Outlines of further classification of audio into finer types and a query-by-example audio retrieval system on top of the coarse classification are also introduced.

Identifying different audio segments in videos is the first step for many important tasks such as event detection and speech transcription. Approaches using Mel-Frequency Cepstral coefficients (MFCCs) with Gaussian mixture models (GMMs) and hidden Markov models (HMMs) perform reasonably well in stationary conditions but do not scale to a broad range of environmental conditions. This paper focuses on the audio segmentation in broadcast soccer videos into audio classes such as Silence, Speech Only, Speech Over Crowd, Crowd Only and Excited, with an alternative feature set which is simplistic as well as robust to changes in the environment conditions. Support Vector Machines (SVMs), Neural Networks and Random Forest are used for the classification. The accuracy achieved with SVMs, Neural Networks and Random Forest are 83.80%, 86.07%, and 88.35% respectively. The proposed features and Random Forest classifier are found to achieve better accuracy compared to the other classifiers.

This paper investigates the issue of automatic segmentation of speech recordings for Broadcast News (BN) and Broadcast Conversation (BC) speech recognition. Our previous segmentation algorithm often exhibited high deletion errors, where some speech segments were misclassified as noises and thus were never passed on to the recognizer. In contrast with our previous segmentation models, which only differentiated between speech and non-speech segments, phonetic knowledge is applied to elaborate speech segments by using multiple models for different types of speech segments. Moreover, the “pronunciation” of the speech segment has been modified to loosen the minimum duration constraint of 540ms long. This method makes use of language specific knowledge, while keeping the number of models low to achieve fast segmentation. Experimental results show that the new segmenter outperforms our previous segmenter significantly, particularly in reducing deletion errors.

Although the UA-Speech and TORGO databases of control and dysarthric speech are invaluable resources made available to the research community with the objective of developing robust automatic speech recognition systems, they have also been used to validate a considerable number of automatic dysarthric speech classification approaches. Such approaches typically rely on the underlying assumption that recordings from control and dysarthric speakers are collected in the same noiseless environment using the same recording setup. In this paper, we show that this assumption is violated for the UA-Speech and TORGO databases. Using voice activity detection to extract speech and non-speech segments, we show that the majority of state-of-the-art dysarthria classification approaches achieve the same or a considerably better performance when using the non-speech segments of these databases than when using the speech segments. These results demonstrate that such approaches trained and validated on the UA-Speech and TORGO databases are potentially learning characteristics of the recording environment or setup rather than dysarthric speech characteristics. We hope that these results raise awareness in the research community about the importance of the quality of recordings when developing and evaluating automatic dysarthria classification approaches.

In this paper, microcontroller based automatic door open system has been developed. The system is developed as speech recognition circuit where programmable voice is used as reference. Programmable means trained voice has been used for identification of authorized and unauthorized person. As software part, MATLAB GUI interface has been used to record authorized voice, and to synthesize the recorded voice with feature extraction such as power spectral density of peak frequency by FFT (Fast Fourier Transform). Then these features will be verified with every incoming voice. This voice activity detection (VAD) is designed and developed to divide the speech signal into speech segment and non-speech segments. As hardware part, ARDUINO microcontroller has been used to implement the circuitry for this well trained automatic security system. This system is with low cost and well-performed in less noisy environment. Accuracy has been obtained around 99%. This system can also be applicable for other controlled access services like voice based biometrics, database access services, voice based dialing, voice mail and remote access to the computers.

A new scheme for nonspeech rejection is proposed by considering that most nonspeech segments do not have well-defined prosodic structures as speech segments do. Certain parameters characterizing the smoothness of the peak index series and of the peak amplitude series of the normalized autocorrelation function are used to make nonspeech segment rejection decisions. The receiver-operating-characteristics curve and recognition word-error-rate reduction measures show that our approach is more effective than garbage-model-based schemes when used in telephone speech recognition.

An efficient noise robust feature is presented to track the speech activity in noisy environments. Speech is modeled by one class of 16 phone-like Gaussian mixtures while noises are modeled by 15 classes of 6 mixtures each. The feature vector used is a concatenation of carefully selected coefficients from MFCC, LPCC, and their first and second derivatives. A finite state machine and energy validation components are proposed as post-processor for the GMM classifier to rectify the misclassified speech segments. The demonstrated speech activity detection system based on our feature detects reliably both speech and non-speech segments. The designed frame work has been benchmarked against the commercially available codecs G.729, GSM-EFR, MR1, and MR2. Results show the proposed technique outperforms all these commonly used techniques under various SNR levels and in different noisy environments.

Introduction to Audio Analysis: A MATLAB Approach

This paper presents an ultra-low-power voice activity detection (VAD) system to discriminate speech from non-speech parts of audio signals. The proposed VAD system uses level-crossing sampling for voice activity detection. The useless samples in the non-speech parts of the signal are eliminated due to the activity-dependent nature of this sampling scheme. A 40 ms moving window with a 30 ms overlap is exploited as a feature extraction block, within which the output samples of the level-crossing analog-to-digital converter (LC-ADC) are counted as the feature. The only variable used to distinguish speech and non-speech segments in the audio input signal is the number of LC-ADC output samples within a time window. The proposed system achieves an average of 91.02% speech hit rate and 82.64% non-speech hit rate over 12 noise types at −5, 0, 5, and 10 dB signal-to-noise ratios (SNR) over the TIMIT database. The proposed system including LC-ADC, feature extraction, and classification circuits was designed in 0.18 µm CMOS technology. Post-layout simulation results show a power consumption of 394.6 nW with a silicon area of 0.044 mm2, which makes it suitable as an always-on device in an automatic speech recognition system.

One major difficulty in conversational research is the time required to segment and transcribe conversational recordings. While recent advances have improved automatic speech recognition technologies, one limitation of current tools is that they are generally catered toward speech that occurs in monologues rather than conversation. Accordingly, the purpose of this project was to develop and validate an automated user-friendly tool for transcribing conversations. This tool, called Autoscribe, converts dyadic conversational audio recordings into Praat TextGrids with time-aligned turn boundaries between speech and non-speech segments and transcripts of all spoken dialogue output. Here we describe the development of this tool as well as its validation on two conversational corpora. Results showed that Autoscribe decreased the amount of active working time needed for TextGrid creation by over 70%. Average transcription accuracy was 92% and average utterance boundary placement of 95%. Thus, Autoscribe affords a practical research tool that drastically reduces the time and resource intensitivity needed for conversational segmentation and transcription.

In this paper, we address the problem of multiple view data fusion in the presence of noise and interferences. Recent studies have approached this problem using kernel methods, by relying particularly on a product of kernels constructed separately for each view. From a graph theory point of view, we analyze this fusion approach in a discrete setting. More specifically, based on a statistical model for the connectivity between data points, we propose an algorithm for the selection of the kernel bandwidth, a parameter that, as we show, has important implications on the robustness of this fusion approach to interferences. Then, we consider the fusion of audio-visual speech signals measured by a single microphone and by a video camera pointed to the face of the speaker. Specifically, we address the task of voice activity detection, i.e., the detection of speech and nonspeech segments, in the presence of structured interferences such as keyboard taps and office noise. We propose an algorithm for voice activity detection based on the audio-visual signal. Simulation results show that the proposed algorithm outperforms competing fusion and voice activity detection approaches. In addition, we demonstrate that a proper selection of the kernel bandwidth indeed leads to improved performance.

Presence of additive noise in speech signals deteriorates the performance of automatic speech recognition systems in cars. For a speech recognition system, we must know where speech and nonspeech segments are. In this paper a new Band Partitioning Spectral Entropy endpoint detection (BPSE) method is used to get the speech start and end point of speech precisely. After that Band Spectral Subtraction (BSS) methods provide in this paper can decrease additive noise obviously. Mel Frequency Cepstral Coefficients (MFCC) are extracted from segmented speech signals. The coefficients are recognized by Hidden Markov Model. The results show that the recognition accuracy can be improved from 39.3% to 95.5%.