Comparing Multi-Scale and Pipeline Models for Speaker Change Detection

Combined speech enhancement and auditory modelling for robust distributed speech recognition

Early in-trouble student identification in an academic credit system is a challenging popular task in the educational data mining field. Only the first few semesters of the students can be observed for the task so that the in-trouble students can be recognized soon and have enough time for improving their study performance. The task can be tackled with different machine learning approaches. In this paper, we use the unsupervised learning approach to determine those students with higher effectiveness and no preparation of other labeled data sets. In this approach, a temporal cluster analysis method is proposed in our work based on the temporal clusters returned by dynamic topic models. In addition, we consider temporal characteristics in the study performance of each student to form a pattern from the temporal clusters he/she belongs to over the time. Similar students share similar patterns and therefore, allowing us to determine the pattern types of in-trouble students and recognize them more accurately. In an evaluation study, experimental results show that our method outperforms the other unsupervised and supervised learning methods with higher Recall and F-measure values. It also obtains better temporal clusters with dynamic topic modeling. As a result, our method is suitable for the early in-trouble student identification task.

This paper addresses the problem of real time speaker change detection and speaker tracking in broadcasted news video analysis. In such a case, both speaker identities and number of speakers are assumed unknown. A two-step speaker change detection algorithm, including potential change detection and refinement, is proposed. Speaker tracking is performed based on the results of speaker change detection. A Bayesian Fusion method is used to fuse multiple audio features to get a more reliable result. The algorithm has low complexity and runs in real-time with a very limited delay in analysis. Our experiments show that the algorithms produce very satisfactory results.

The conventional speaker change detection (SCD) method using Bayesian Information Criterion (BIC) has been widely used. However, its performance relies on the choice of penalty factor and suffers from mass calculation. The twostep SCD is less time consuming but generates more detection errors. The limitation of conventional method’s performance originates from the two adjacent data windows. We propose a strategy that inserts an interval between the two adjacent fixed-size data windows in each analysis window. The dissimilarity value between the data windows is regarded as the probability of a speaker identity change within the interval area. Then this analysis window is slid along the audio by a large step to locate the areas where speaker change points may appear. Afterwards we only focus on these areas and locate precisely where the change points are. Other areas where a speaker change point unlikely appears are abandoned. The proposed method is computationally efficient and more robust to noise and penalty factor compared with conventional method. Evaluated on the corpus of China Central Television (CCTV) news, the proposed method obtains 74.18% reduction in calculation time and 22.24% improvement in F1-measure compared with the conventional approach.

Speaker change detection in casual conversations using excitation source features

While machine learning and artificial intelligence have long been applied in networking research, the bulk of such works has focused on supervised learning. Recently there has been a rising trend of employing unsupervised machine learning using unstructured raw network data to improve network performance and provide services such as traffic engineering, anomaly detection, Internet traffic classification, and quality of service optimization. The interest in applying unsupervised learning techniques in networking emerges from their great success in other fields such as computer vision, natural language processing, speech recognition, and optimal control (e.g., for developing autonomous self-driving cars). Unsupervised learning is interesting since it can unconstrain us from the need of labeled data and manual handcrafted feature engineering thereby facilitating flexible, general, and automated methods of machine learning. The focus of this survey paper is to provide an overview of the applications of unsupervised learning in the domain of networking. We provide a comprehensive survey highlighting the recent advancements in unsupervised learning techniques and describe their applications for various learning tasks in the context of networking. We also provide a discussion on future directions and open research issues, while also identifying potential pitfalls. While a few survey papers focusing on the applications of machine learning in networking have previously been published, a survey of similar scope and breadth is missing in literature. Through this paper, we advance the state of knowledge by carefully synthesizing the insights from these survey papers while also providing contemporary coverage of recent advances.

An efficient scheme, based on the Bayesian information criterion (BIC), for the detection of speaker changes in an audio stream is introduced and investigated. BIC has been the subject of considerable attention in recent years due to its effectiveness for speaker change detection (SCD) as well as the detection of other forms of acoustic changes. A main difficulty in BIC-based SCD has been reported to be that of the computational complexity. The scheme proposed here tackles this problem by reducing the computational load in the previously proposed algorithms significantly, without compromising their effectiveness. The paper describes the new scheme thoroughly and analyses its performance. Experiments are based on 3 hours of broadcast news with 416 speaker changes. With this data, the proposed scheme has been found to be capable of running in about 0.06 times real-time whilst keeping the rate of each of misdetection and false alarm close to 9%.

Speaker change detection (SCD) is an important task in dialog modeling. Our paper addresses the problem of text-based SCD, which differs from existing audio-based studies and is useful in various scenarios, for example, processing dialog transcripts where speaker identities are missing (e.g., OpenSubtitle), and enhancing audio SCD with textual information. We formulate text-based SCD as a matching problem of utterances before and after a certain decision point; we propose a hierarchical recurrent neural network (RNN) with static sentence-level attention. Experimental results show that neural networks consistently achieve better performance than feature-based approaches, and that our attention-based model significantly outperforms non-attention neural networks.

We describe an audio-visual automatic continuous speech recognition system, which significantly improves speech recognition performance over a wide range of acoustic noise levels, as well as under clean audio conditions. The system utilizes facial animation parameters (FAPs) supported by the MPEG-4 standard for the visual representation of speech. We also describe a robust and automatic algorithm we have developed to extract FAPs from visual data, which does not require hand labeling or extensive training procedures. The principal component analysis (PCA) was performed on the FAPs in order to decrease the dimensionality of the visual feature vectors, and the derived projection weights were used as visual features in the audio-visual automatic speech recognition (ASR) experiments. Both single-stream and multistream hidden Markov models (HMMs) were used to model the ASR system, integrate audio and visual information, and perform a relatively large vocabulary (approximately 1000 words) speech recognition experiments. The experiments performed use clean audio data and audio data corrupted by stationary white Gaussian noise at various SNRs. The proposed system reduces the word error rate (WER) by 20% to 23% relatively to audio-only speech recognition WERs, at various SNRs (0–30 dB) with additive white Gaussian noise, and by 19% relatively to audio-only speech recognition WER under clean audio conditions.

In this paper we propose an unsupervised speaker change detection (SCD) system developed for mobile device applications. We use Bayesian information criterion (BIC) to find initial speaker changes, which are then verified or discarded in the second phase by utilizing modified BIC and silence detector information. Silence information usage after initial BIC in decision making is useful to separate real changes from noise peaks. Enhanced peak detector adjusts BIC penalty parameter automatically, which improve the robustness and feasibility. Improved BIC based false alarm compensation (FAC) merges effectively consecutive segments belonging to same speaker. Our experiments have shown the robustness of the algorithm and it produces very satisfactory results for difficult mobile phone recorded speech data.

The development of tools for automated analysis of data collected for mine countermeasures (MCM) search, classify and map (SCM) has predominantly focused on automated target recognition for individual mines. As MCM target detection becomes more automated, detecting spatial patterns in the positioning of mines should follow. Natural restrictions on mine deployment such as depth and bottom contour as well as mine deployment methodology and protocol effectively create mine line patterns that are typical to coastal areas. We have developed a mine line detection algorithm that uses nonparametric spatial analysis to identify mine-like contacts (MLCs) that fit into mine line patterns dictated by standard mine laying protocol in coastal regions. In general, a set of MLCs reported from a mine detection sensor will consist of both actual mines and false alarms; the algorithm classifies every MLC in this set into one of three classes: fixed — the MLCψfits into a mine line pattern with fixed mine spacing; random -the MLCψfits into a mine line pattern with random mine spacing; or false alarm-ψ-the MLCψdoes not fit into a mine line pattern and is subsequently not considered a mine. The latest version of the algorithm utilizes nonparametric techniques and requires no knowledge of either the detection sensor's probability of detection (P d ) or probability of false alarm (P fa ). A nonparametric anomaly detector first identifies regions with unusually high MLC density for more detailed analysis. Suspected regions are subsequently analyzed for unusually regular patterns of MLCs. The probabilities that the observed target densities and target patterns could occur simply by chance are estimated by comparison with results from nearby areas using basic information about mine deployment. To ensure that a mine line's full extent is captured, the algorithm attempts to grow a line after it is detected. And a more detailed probability analysis distinguishes random mine lines from fixed-spaced mine lines. Extensive Matlab simulations characterize the algorithm's performance across a wide range of operating environments and simulated minefield configurations. In total, ten replications each of 1,367 different configurations have been evaluated. These configurations include three different water depths (10 ft., 6 ft. and 200 ft.), six false notional alarm rates (1, 2,5,10,15 and 20 false alarms per sq. NM), three false alarm distributions (uniform, sloped, and striped), three sensor probabilities of detections (0.4, 0.6 and 0.9), three target confidence scores (0.6,0.8 and 1.0) and four false alarm confidence scores (0.5, 0.6, 0.8 and 1.0). The range of simulation parameters was selected in part as an attempt to push the algorithm past its acceptable performance limits. However, the algorithm performed remarkably well even under the worst conditions. For MLC sets with false alarm rates as high as 20 per square nautical mile, the algorithm correctly classifies over 98% of the MLCs as mines and detects 97% of the mine lines. Furthermore, it reduces the false alarm rate from 20 to 0.2 per square nautical mile with a false mine line rate of 0.14 false lines per square nautical mile. The algorithm achieves even lower false alarm rates when the initial false alarm rate is less than 20. The automated mine line detection discovers many obscure lines that are virtually undetectable to the naked eye.

We herein propose a domain-independent topic segmentation algorithm for free-form multi-party meeting recordings. The advantage of the proposed algorithm is that topical and lexical knowledge, which are difficult to adapt to the target meeting before speech recognition and topic segmentation, are not required. For an errorful sequence of phonemes obtained using a continuous phoneme recognizer, the proposed algorithm exhaustively analyzes the occurrence pattern of subsequences of phonemes and partitions the sequence into segments with coherent patterns. An empirical study on the ICSI Meeting Corpus has indicated that it performs comparably to lexical-cohesion-based text segmenters applied to human transcripts. Furthermore, the performance of the text segmenters applied to LVCSR output decreases significantly when keywords are not included in the lexicon. This suggests that, for the purpose of obtaining topical structure, the phoneme sequence segmenter could be more robust than text segmenters with LVCSR.

Unsupervised learning approaches are used in various applications such as speech recognition, image compression, information retrieval and activity recognition. This paper introduces a novel unsupervised approach for clustering multi-dimensional time series that present the 3-d acceleration data measured with body-worn accelerometers. More specifically, the proposed approach uses a statistical model based on Multiple Hidden Markov Model Regression (MHMMR) to automatically analyze the human activity. This method takes into account the sequential appearance and temporal evolution of the data to easily detect static and dynamic activities. Comparisons with existing unsupervised approaches, including the standard Gaussian Mixture Model, the k-means algorithm, the DBSCAN algorithm and the standard HMM, demonstrate the effectiveness of the proposed approach.

We present a new algorithm for time series prediction using temporal fuzzy clustering. The algorithm is based on the framework of temporal clustering that was applied successfully to analyze, segment and recognize patterns of nonstationary signals in applications such as speech recognition and biomedical signal analysis. We combine fuzzy clustering in the observation space and cluster validation in the time axis in order to generate a prediction according to the online estimation of a time varying multivariate mixture distribution function that underlies the series elements. The resulting temporal behavior of the membership matrices can also be used to extract a prediction on the future probability distribution function (PDF) of the time series. The algorithm is more feasible than common methods such as hidden Markov models (HMM) in predicting nonstationary signals with a slow drift in their PDF and is also more efficient from a computation standpoint.

The estimation of the instantaneous frequency (IF) of a harmonic complex-valued signal with an additive noise using the Wigner distribution is considered. If the IF is a nonlinear function of time, the bias of the estimate depends on the window length. The optimal choice of the window length, based on the asymptotic formulae for the variance and bias, can be used in order to resolve the bias-variance tradeoff. However, the practical value of this solution is not significant because the optimal window length depends on the unknown smoothness of the IF. The goal of this paper is to develop an adaptive IF estimator with a time-varying and data-driven window length, which is able to provide quality close to what could be achieved if the smoothness of the IF were known in advance. The algorithm uses the asymptotic formula for the variance of the estimator only. Its value may be easily obtained in the case of white noise and relatively high sampling rate. Simulation shows good accuracy for the proposed adaptive algorithm.