Method and apparatus for continuous speech recognition using a layered, self-adjusting decoder network

An apparatus and method for selective distributed speech recognition includes a dialog manager (104) that is capable of receiving a grammar type indicator (170). The dialog manager (104) is capable of being coupled to an external speech recognition engine (108), which may be disposed on a communication network (142). The apparatus and method further includes an audio receiver (102) coupled to the dialog manager (104) wherein the audio receiver (104) receives a speech input (110) and provides an encoded audio input (112) to the dialog manager (104). The method and apparatus also includes an embedded speech recognition engine (106) coupled to the dialog manager (104), such that the dialog manager (104) selects to distribute the encoded audio input (112) to either the embedded speech recognition engine (106) or the external speech recognition engine (108) based on the corresponding grammar type indicator (170).

A speech recognizer suitable for distributed speech recognition is robust to missing speech feature vectors. Speech is transmitted via a packet switched network in the form of basic feature vectors. Missing feature vectors are detected and replacement feature vectors are estimated by interpolation of received data prior to speech recognition. Features may be converted and interpolation may be accomplished in a spectral domain.

Speech parameters (Ph and Pl) are derived for consonant classification and recognition by separating a speech signal into Low and High frequency bands, then in each band obtaining the time first-derivative, from which the min-max differences (power dip) are obtained (Ph and Pl). The distribution of Ph and Pl in a two-dimensional plot for a discriminant diagram classifies the consonant phoneme.

Speech signal analysis for data reduction, as stored for synthesis or recognition, is improved by features including: digital spectral analysis; reduction of channel data and bit allocation by selective summation of groups of contiguous data; using the mean average of the log amplitude to find the deviation for each channel; also using the instaneous shape of the mean value for each channel for pairs of adjacent frames, all combined to find a feature ensemble for each pair of adjacent frames.

First Page

Phonetic recognition is provided by capturing dynamical behavior and statistical dependencies of the acoustic attributes used to represent a subject speech waveform. A segment based framework is employed. Temporal behavior is modelled explicitly by creating dynamic templates, called tracks, of the acoustic attributes used to represent the speech waveform, and by generating the estimation of the acoustic spatio-temporal correlation structure. An error model represents this estimation as the temporal and spatial correlations between the input speech waveform and track generated speech segment. Models incorporating these two components (track and error estimation) are created for both phonetic units and for phonetic transitions. Phonetic contextual influences are accounted for by merging context-dependent tracks and pooling error statistics over the different contexts. This allows for a large number of contextual models without compromising the robustness of the statistical parameter estimates. The transition models also supply contextual information.

First Page

In a word speech recognition method which performs pattern matching between unknown speech pattern and multiple reference templates and detects that one of the reference templates which provides the smallest one of distance measures detected between the unknown speech pattern and the reference templates, when the difference d between the speech period length of the unknown speech pattern and the speech period length of a selected reference template exceeds a fixed threshold value e1, partial patterns are extracted from the unknown speech pattern, each starting at a different position, and the minimum one of the distances obtained by pattern matching between these extracted partial patterns and the selected reference template is determined to be the distance between the selected reference template and the unknown speech pattern. When the difference d is in the range of -e2 ≦d≦e1, pattern matching is performed between speech periods of the unknown speech pattern and the reference templates with their variation periods eliminated therefrom at their both ends.

Speech recognition is achieved using a normalized cumulative distance. A normalized Dynamic Programming (DP) value is calculated by dividing a cumulative path distance by an optimal integral path length. The path length is calculated iteratively by adding 2 if the warping path is diagonal or by adding 3 if the warping path is horizontal or vertical. Distance may be calculated by measuring a difference between input power and average power. The power difference is weighted by a coefficient (λ) between 0 and 1. A Mahalanobis distance is then weighted by (1-λ) and added to the weighted power difference.

A system is disclosed for recognizing a pattern in a collection of data given a context of one or more other patterns previously identified. Preferably the system is a speech recognition system, the patterns are words and the collection of data is a sequence of acoustic frames. During the processing of each of a plurality of frames, for each word in an active vocabulary, the system updates a likelihood score representing a probability of a match between the word and the frame, combines a language model score based on one or more previously recognized words with that likelihood score, and prunes the word from the active vocabulary if the combined score is below a threshold. A rapid match is made between the frames and each word of an initial vocabulary to determine which words should originally be placed in the active vocabulary. Preferably the system enables an operator to confirm the system's best guess as to the spoken word merely by speaking another word, to indicate that an alternate guess by the system is correct by typing a key associated with that guess, and to indicate that neither the best guess nor the alternate guesses was correct by typing yet another key. The system includes other features, including ones for determining where among the frames to look for the start of speech, and a special hardware processor for computing likelihood scores.

The present invention comprises a method and system for speech recognition. The method includes the steps of receiving digital data representation of speech having at least one word, which includes a waveform representation of the at least one word. The method includes searching a library containing digital data representation of a plurality of words and matching the digital data representation of the at least one word to digital data representation of a word in the library. The method further includes the steps of determining the number of syllables in the digital data representation of the at least one word, and then providing an ASCII representation of the matched word in response to the number of syllables of the at least one word being identical to the number of syllables of the matched word in the library.

Statistical acoustic processing and apparatus for speech recognition using a toned phoneme system

A method of speech recognition with compensation is provided by modifying HMM models trained on clean speech with cepstral mean normalization. For all speech utterances the MFCC vector is calculated for the clean database. This mean MFCC vector is added to the original models. An estimate of the background noise is determined for a given speech utterance. The model mean vectors adapted to the noise are determined. The mean vector of the noisy data over the noisy speech space is determined and this is removed from model mean vectors adapted to noise to get the target model.

Existing Parkinson's disease (PD) speech recognition methods are predominantly based on fixed-text speech recognition and are specifically designed for a particular language, which limits their generalizability. To address this issue, this paper proposes a domain generalization method, termed Domain-Adversarial Representation Learning (DARL), for recognizing PD continuous speech. Specifically, DARL extracts domain-invariant features by jointly optimizing classification loss and distribution discrepancy loss, utilizing labeled fixed-text speech data and unlabeled target continuous speech data. By narrowing the domain gap between fixed-text and continuous speech data, the proposed approach significantly improves the model generalization capability across different languages. Experimental results on two tasks and multiple evaluation metrics consistently demonstrate the superior performance of the proposed method in PD continuous speech recognition.

A words phonetic decoding method in automatic speech recognition is considered. The properties of Kullback–Leibler divergence are used to synthesize the estimation of the distribution of divergence between minimum speech units (e.g., single phonemes) inside a single class. It is demonstrated that the minimum variance of the intraphonemic divergence is reached when the phonetic database is tuned to the voice of a single speaker. The estimations are proven by experimental results on the recognition of vowel sounds and isolated words of Russian language.