Modeling musical sounds with an interpolating state model

With the fast booming of online music repositories, there is a need for content-based automatic indexing which will help users to their favorite music objects in real time. Recently, numerous successful approaches on musical data feature extraction and selection have been proposed for instrument recognition in monophonic sounds. Unfortunately, none of these methods can be successfully applied to polyphonic sounds. Identification of music instruments in polyphonic sounds is still difficult and challenging, especially when harmonic partials are overlapping with each other. This has stimulated the research on music sound separation and new features development for content-based automatic music information retrieval. Our goal is to build a cooperative query answering system (QAS), for a musical database, retrieving from it all objects satisfying queries like find all musical pieces in pentatonic scale with a viola and piano where viola is playing for minimum 20 seconds and piano for minimum 10 seconds. We use the database of musical sounds, containing almost 4000 sounds taken from the MUMs (McGill University Master Samples), as a vehicle to construct several classifiers for automatic instrument recognition. Classifiers showing the best performance are adopted for automatic indexing of musical pieces by instruments. Our musical database has an FS-tree (Frame Segment Tree) structure representation. The cooperativeness of QAS is driven by several hierarchical structures used for classifying musical instruments.

A new generic nonlinear feature extraction map f is presented based on concepts from approximation theory. Let f map an input data vector x/spl isin//spl Rfr//sup n/, where n is high, to an appropriate feature vector y/spl isin//spl Rfr//sup m/, where m is sufficiently low. Also let X={X/sub 1/,...,X/sub N/} denote an available training set in /spl Rfr//sup n/. In this paper f is derived by requiring that the geometric structure (metric space attributes) of the points f(X)={f(X/sub 1/),...,f(X/sub N/)} in the feature space /spl Rfr//sup m/ be as similar as possible to the structure of the points X in the data space /spl Rfr//sup n/. This is accomplished by selecting first an appropriate dimension m for the feature space /spl Rfr//sup m/ according to the size N of the available training set X, subject to bounds on the distortion of the data structure caused by f and on the error in the estimation of the underlying likelihood functions in the feature space. The map f(i) is designed by a multi-dimensional scaling (MDS) approach that minimizes the Sammon's cost function. This approach uses graph-theoretic (minimal spanning tree) and genetic algorithmic concepts to search efficiently the optimal structure-preserving point-to-point mapping of the training samples X/sub 1/,...,X/sub N/ to their images in /spl Rfr//sup m/,Y/sub 1/,...,Y/sub N/. Finally, an optimal interpolating (OI) artificial neural network is used to recover the entire function f:/spl Rfr//spl rarr//spl Rfr//sup m/ by interpolating the values y/sub i/=1,...,N at X/sub i/, i=1,...,N. Preliminary simulation results based on this approach are also given.

This paper describes an off-line segmentation-free handwritten Arabic words recognition system. The described system uses discrete HMMs with explicit state duration of various kinds (Gauss, Poisson and gamma) for the word classification purpose. After preprocessing, the word image is analyzed from right to left in order to extract from it a sequence of feature vectors. Then, vector quantization is applied to this sequence and its output is submitted to a HMMs classifier based on a likelihood criterion for identifying the word using the viterbi algorithm. Several experiments were performed using the IFN/ENIT benchmark database, they showed, on the one hand, a substantial improvement in the recognition rate when HMMs with explicit state duration of either discrete or continuous distribution are used instead of classical HMMs (i.e. with implicit state duration), on the other hand, the gamma distribution for the state duration, that have given the best recognition rate (91.23 % in top 2), seems more suitable for the HMMs based modeling of Arabic handwriting

An apparatus and method of quantizing a sequence of input data vectors using delayed decision switched prediction and vector quantization. The method has the following steps of operation: (a) predicting a next vector element from said sequence of input data vectors to generate a set of prediction vectors; (b) subtracting the set of prediction vectors from the next vector element to generate a set of prediction error vectors; (c) multi-stage vector quantizing the set of prediction error vectors to generate a set of quantized prediction error vectors with each of the stages having at least one of the tables and local decision means to generate a final quantization error vector according to a predetermined distance measure; (d) selecting one predictor out of the set of predictors from the switched prediction step and selecting, for each of the stages, at least one entry from the set of tables of the vector quantization step according to the predetermined distance measure, generating a quantized data vector.

In this paper we present analog current mode Euclidean distance calculation (EDC) block, which calculates the distance between two current vectors. The proposed circuit is an important part of the CMOS-implemented Kohonen’s neural network (KNN) designed for medical applications. The input data vector is compared with the weights’ vector in each neuron in proposed KNN. The neuron, whose weights are the closest to the input training vector becomes the winner and in the next step changes its weights. Proposed EDC block performs several operations such as: subtraction, squaring and summing of the current signals. The output current is the exact measure of the Euclidean distance. Proposed circuit dissipates power 15 μW from 1.5 V voltage supply, working with 20 MHz input data frequency. The signal frequency as well as the power dissipation may be scaled down to 1 MHz and 300 nA. Proposed EDC circuit, in CMOS 0.18 μm technology, occupies area about 500 μm2.

This paper describes the architecture design of novel massively parallel self-organizing map (SOM) neural networks. The proposed architecture, referred to as the planar SOM (PSOM), is described as a soft IP core synthesized in VHDL. The SOM neural network’s size and the input data vectors’ dimension are adjustable parameters. In this work, several SOM architectures are synthesized and their performance is evaluated for Xilinx Virtex-7 FPGAs. The presented hardware architecture allows online learning and can be easily adapted to a large variety of SOM topologies without a considerable design effort. A [Formula: see text] SOM hardware is validated through the FPGA implementation and its performances with an estimated working frequency of 297[Formula: see text]MHz for a 23-element input vector will reach 21,970 MCUPS in the learning phase and 35,902 MCPS in the recall one.

This paper presents a new countermeasure for the protection of automatic speaker verification systems from spoofed, converted voice signals.The new countermeasure is based on the analysis of a sequence of acoustic feature vectors using Local Binary Patterns (LBPs).Compared to existing approaches the new countermeasure is less reliant on prior knowledge and affords robust protection from not only voice conversion, for which it is optimised, but also spoofing attacks from speech synthesis and artificial signals, all of which otherwise provoke significant increases in false acceptance.The work highlights the difficulty in detecting converted voice and also discusses the need for formal evaluations to develop new countermeasures which are less reliant on prior knowledge and thus more reflective of practical use cases.

Speech spoofing countermeasures based on source voice analysis and machine learning techniques

Continuous density hidden Markov models (CD-HMMs) are an essential component of modern systems for automatic speech recognition (ASR). These models assign probabilities to the sequences of acoustic feature vectors extracted by signal processing of speech waveforms. In this chapter, we investigate a new framework for parameter estimation in CD-HMMs. Our framework is inspired by recent parallel trends in the fields of ASR and machine learning. In ASR, significant improvements in performance have been obtained by discriminative training of acoustic models. In machine learning, significant improvements in performance have been obtained by discriminative training of large margin classifiers. Building on both these lines of work, we show how to train CD-HMMs by maximizing an appropriately defined margin between correct and incorrect decodings of speech waveforms. We start by defining an objective function over a transformed parameter space for CD-HMMs, then describe how it can be optimized efficiently by simple gradient-based methods. Within this framework, we obtain highly competitive results for phonetic recognition on the TIMIT speech corpus. We also compare our framework for large margin training to other popular frameworks for discriminative training of CD-HMMs.

Color-based indexing is an important tool in image database retrieval. Compared with other features of the image, color features are less sensitive to noise and background complication. Based on the human visual system's perception of color information, this paper presents a dependent scalar quantization approach to extract the characteristic colors of an image as color features. The characteristic colors are suitably arranged in order to obtain a sequence of feature vectors. Using this sequence of feature vectors, a dynamic matching method is then employed to match the query image with database images for a nonstationary identification environment. The empirical results show that the characteristic colors are reliable color features for image database retrieval. In addition, the proposed matching method has acceptable accuracy of image retrieval compared with existing methods.

Intuitionistic Fuzzy C-means (IFCM) is a robust clustering method which is based upon intuitionistic fuzzy set theory. It uses Euclidean distance as a distance metric, hence can only cluster hyper spherically distributed data-sets in data space or in feature space. FCM and KFCM with a new distance measure (FCM-σ and KFCM-σ) can detect non-hyperspherical clusters in data space and feature space but they are sensitive to noise and produce inefficient results in the presence of noise. This paper present a robust Intuitionistic Fuzzy c-means(IFCM-σ) and a robust kernel Intutitionistic Fuzzy C-Means(KIFCM-σ) with a new distance metric that incorporates the distance variation in a cluster to regularize the distance between data point and the cluster centroid. Propose algorithms are the hybridization of IFCM, kernel function, and new distance metric in the data space and in the feature space which avoid various problems of IFCM and FCM-σ. Experiments are done using two-dimensional synthetic data-sets and noisy digital images, and results are compared with IFCM, KIFCM, FCM-σ and KFCM-σ. The results show that our proposed algorithms, especially KIFCM-σ are more effective.

In video compression, some kernel functions such as block matching, discrete wavelet transform, vector quantization, etc., are of prime essential, but with a large amount of computation. Recently, many systolic array architectures have been designated for performing each of those functions in real time. In fact, many kernel functions contain the similar computational procedure. If we dissect these functions into the basic matrix-vector product forms, a unified design for them becomes feasible. In this brief, by carefully extracting the common computation component, a unified one-dimensional systolic array design that can perform at least the above three functions is presented. In this design, the input data is serial-in to save the amount of pins required, and the data flow are carefully arranged to simplify the interconnection between computation components. When 64 registers are in the on-chip memory, our design can perform three typical functions: (1) the full-search block matching with block size 16/spl times/16 and the search range (-8,7); (2) the 2-D Harr transform with block size 8/spl times/8; and (3) the vector quantization with input vector size 4/spl times/4 and codebook size 256. Since the unified architecture reduces hardware costs, and has a regular hardware structure, it is suited for VLSI implementation for video/image compression applications that require all the functions.

Facial expression provides an important clue for teachers to know the learning status of students. Thus, vision-based expression analysis is valuable not only in Human-Computer Interface but also in e-Learning. We propose a computer vision system to automatically analyze learners' video to recognize nonverbal facial expressions to discover learning status of students in distance education. In the first stage, Adaboost classifiers are applied to extract candidates of facial parts. Then spatial relationships are utilized to determine the best combination of facial features to form a feature vector. In the second stage, each feature vector sequence is trained and recognized as a specific emotional expression using Hidden Markov Model (HMM). The estimated probabilities of six expressions are combined into an expression vector. The last stage is to analyze the expression vector sequence to figure out the learning situation of the student. Gaussian Mixture Model (GMM) is applied to evaluate three learning scores (Understanding, Interaction, and Consciousness) that are integrated into a status vector. Each evaluated status vector reflects the learning status of a student and is helpful to not only teachers but also students for improving teaching and learning.

Automatic lipreading is automatic speech recognition that uses only visual information. The relevant data in a video signal is isolated and features are extracted from it. From a sequence of feature vectors, where every vector represents one video image, a sequence of higher level semantic elements is formed. These semantic elements are “visemes” the visual equivalent of “phonemes” The developed prototype uses a Time Delayed Neural Network to classify the visemes.

Following recent advancements in deep learning and artificial intelligence, spoken language identification applications are playing an increasingly significant role in our day-to-day lives, especially in the domain of multi-lingual speech recognition. In this article, we propose a spoken language identification system that depends on the sequence of feature vectors. The proposed system uses a hybrid Convolutional Recurrent Neural Network (CRNN), which combines a Convolutional Neural Network (CNN) with a Recurrent Neural Network (RNN) network, for spoken language identification on seven languages, including Arabic, chosen from subsets of the Mozilla Common Voice (MCV) corpus. The proposed system exploits the advantages of both CNN and RNN architectures to construct the CRNN architecture. At the feature extraction stage, it compares the Gammatone Cepstral Coefficient (GTCC) feature and Mel Frequency Cepstral Coefficient (MFCC) feature, as well as a combination of both. Finally, the speech signals were represented as frames and used as the input for the CRNN architecture. After conducting experiments, the results of the proposed system indicate higher performance with combined GTCC and MFCC features compared to GTCC or MFCC features used individually. The average accuracy of the proposed system was 92.81% in the best experiment for spoken language identification. Furthermore, the system can learn language-specific patterns in various filter size representations of speech files.