Sequence labeling to detect stuttering events in read speech

BackgroundIn biomedical text mining, named entity recognition (NER) is an important task used to extract information from biomedical articles. Previously proposed methods for NER are dictionary- or rule-based methods and machine learning approaches. However, these traditional approaches are heavily reliant on large-scale dictionaries, target-specific rules, or well-constructed corpora. These methods to NER have been superseded by the deep learning-based approach that is independent of hand-crafted features. However, although such methods of NER employ additional conditional random fields (CRF) to capture important correlations between neighboring labels, they often do not incorporate all the contextual information from text into the deep learning layers.ResultsWe propose herein an NER system for biomedical entities by incorporating n-grams with bi-directional long short-term memory (BiLSTM) and CRF; this system is referred to as a contextual long short-term memory networks with CRF (CLSTM). We assess the CLSTM model on three corpora: the disease corpus of the National Center for Biotechnology Information (NCBI), the BioCreative II Gene Mention corpus (GM), and the BioCreative V Chemical Disease Relation corpus (CDR). Our framework was compared with several deep learning approaches, such as BiLSTM, BiLSTM with CRF, GRAM-CNN, and BERT. On the NCBI corpus, our model recorded an F-score of 85.68% for the NER of diseases, showing an improvement of 1.50% over previous methods. Moreover, although BERT used transfer learning by incorporating more than 2.5 billion words, our system showed similar performance with BERT with an F-scores of 81.44% for gene NER on the GM corpus and a outperformed F-score of 86.44% for the NER of chemicals and diseases on the CDR corpus. We conclude that our method significantly improves performance on biomedical NER tasks.ConclusionThe proposed approach is robust in recognizing biological entities in text.

This article addresses language identification at the word level in Indian social media corpora taken from Facebook, Twitter and WhatsApp posts that exhibit code-mixing between English-Hindi, English-Bengali, as well as a blend of both language pairs. Code-mixing is a fusion of multiple languages previously mainly associated with spoken language, but which social media users also deploy when communicating in ways that tend to be rather casual. The coarse nature of code-mixed social media text makes language identification challenging. Here, the performance of deep learning on this task is compared to feature-based learning, with two Recursive Neural Network techniques, Long Short Term Memory (LSTM) and bidirectional LSTM, being contrasted to a Conditional Random Fields (CRF) classifier. The results show the deep learners outscoring the CRF, with the bidirectional LSTM demonstrating the best language identification performance.

Towards subject independent continuous sign language recognition: A segment and merge approach

Alongside acoustic information, linguistic features based on speech transcripts have been proven useful in Speech Emotion Recognition (SER). However, due to the scarcity of emotion labelled data and the difficulty of recognizing emotional speech, it is hard to obtain reliable linguistic features and models in this research area. In this paper, we propose to fuse Automatic Speech Recognition (ASR) outputs into the pipeline for joint training SER. The relationship between ASR and SER is understudied, and it is unclear what and how ASR features benefit SER. By examining various ASR outputs and fusion methods, our experiments show that in joint ASR-SER training, incorporating both ASR hidden and text output using a hierarchical co-attention fusion approach improves the SER performance the most. On the IEMOCAP corpus, our approach achieves 63.4% weighted accuracy, which is close to the baseline results achieved by combining ground-truth transcripts. In addition, we also present novel word error rate analysis on IEMOCAP and layer-difference analysis of the Wav2vec 2.0 model to better understand the relationship between ASR and SER.

This paper presents our approach to participate in the SemEval 2017 Task 12: Clinical TempEval challenge, specifically in the event and time expressions span and attribute identification subtasks (ES, EA, TS, TA). Our approach consisted in training Conditional Random Fields (CRF) classifiers using the provided annotations, and in creating manually curated rules to classify the attributes of each event and time expression. We used a set of common features for the event and time CRF classifiers, and a set of features specific to each type of entity, based on domain knowledge. Training only on the source domain data, our best F-scores were 0.683 and 0.485 for event and time span identification subtasks. When adding target domain annotations to the training data, the best F-scores obtained were 0.729 and 0.554, for the same subtasks. We obtained the second highest F-score of the challenge on the event polarity subtask (0.708). The source code of our system, Clinical Timeline Annotation (CiTA), is available at https://github.com/lasigeBioTM/CiTA.

This paper describes two experiments aimed at exploring the relationship between linguistic aspects and perceived proficiency in read and spontaneous speech. 5,000 utterances of read speech by 50 non-native speakers of Korean in Experiment 1, and of 6,000 spontaneous speech utterances in Experiment 2 were scored for proficiency by native human raters and were analyzed by factors known to be related to perceived proficiency. The results show that the factors investigated in this study can be employed to predict proficiency ratings, and the predictive power of fluency and pitch and accent accuracy is strong for both read and spontaneous speech. We also observe that while proficiency ratings of read speech are mainly related to segmental accuracy, those of spontaneous speech appear to be more related to pitch and accent accuracy. Moreover, proficiency in read speech does not always equate to the proficiency in spontaneous speech, and vice versa, with Pearson’s per-speaker correlation score of 0.535.

We investigate the use of Conditional Random Fields (CRF) to model confusions and account for errors in the phonetic decoding derived from Automatic Speech Recognition output. The goal is to improve the accuracy of approximate phonetic match, given query terms and an indexed database of documents, in a vocabulary independent audio search system. Audio data is ingested, segmented, decoded to produce a sequence of phones, and subsequently indexed using phone N-grams. Search is performed by expanding queries into phone sequences and matching against the index. The approximate match score is derived from a CRF, trained on parallel transcripts, which provides a general framework for modeling the errors that a recognition system may make taking contextual effects into consideration. Our approach differs from other work in the field in that we focus on using CRFs to model context dependent phone level confusions, rather than on explicitly modeling parameters of an edit distance. While, the results we obtain on both in and out of vocabulary (OOV) search tasks improve on previous work which incorporated high order phone confusions, the gains for OOV are more impressive.

In order to improve the performance of a deep-learning neural network, the paper outlines a stack-based approach incorporating various information sources. A named entity recognition system for Amharic was implemented using a recurrent neural network, a bi-directional long short term memory model. Word vectors based on semantic information were built using an unsupervised learning algorithm, word2vec, while a Conditional Random Fields (CRF) classifier was trained on language independent features to predict each token’s named entity class. The predictions, features and word vectors were fed to the deep neural network to assign labels to the words. This stack-based approach reached an 74.26% F-score, outperforming various other deep-learning set-ups, as well as a baseline CRF classifier, and an ensemble method incorporating the same information sources.

The recognition of read and spontaneous speech in local vernacular: The case of Zurich German

This paper deals with an approach to Automatic Language Identification using only prosodic modeling. The actual approach for language identification focuses mainly on phonotactics because it gives the best results. We propose here to evaluate the relevance of prosodic information for language identification with read studio recording (previous experiment [1]) and spontaneous telephone speech. For read speech, experiments were performed on the five languages of the MULTEXT database [2]. On the MULTEXT corpus, our prosodic system achieved an identification rate of 79 % on the five languages discrimination task. For spontaneous speech, experiments are made on the ten languages of the OGI Multilingual telephone speech corpus [3]. On the OGI MLTS corpus, the results are given for languages pair discrimination tasks, and are compared with results from [4]. As a conclusion, if our prosodic system achieves good performance on read speech, it might not take into account the complexity of spontaneous speech prosody. 1.

Text simplification is a vital work for comprehending patent claims due to its complex syntactic structures and lengthy sentences. Therefore, almost all patent analysis practitioners cannot be able to directly and intuitively understand patent essence even through some common natural language processing (NLP) tools are applied to parse these patent claim paragraph or sentences. Universal text analysis tools above is almost useless, or even crashed when applied to some complex paragraphs of patent claims. Therefore, it is necessary to propose a patent text oriented simplification approach to help patent researchers grasp the essence of patent quickly and intuitively. Motivated by the above reason, we in this article propose a simplification method based on deep learning to segment patent claim into shorter and comprehensible sentences for downstream tasks of patent analysis. The proposed approach contains two stages: on one stage, we use a machine learning approach of conditional random field (CRF) to decompose syntactically complex paragraphs into coarse‐grained level sentences with simplified structures and complete semantics; on another stage, a deep Learning architecture of bidirectional long‐short term memory (Bi‐LSTM)‐CRF is applied to segment coarse‐grained and lengthy sentences of former stage into fined‐grained and shorter sentences. Compared with a series of baselines, our patent segmentation architecture based on deep learning of Bi‐LSTM‐CRF achieves higher performance than any other methods on the evaluation measures of precision, recall, and F1.

Semantic Role Labeling (SRL) is one of the most challenging tasks in Natural Language Processing (NLP). SRL is a task that consists of predicate identification, argument identification and argument classification. In this article we present novel approach for argument classification that is based on deep neural network architecture. Traditional discrete based SRL relies heavily on feature engineering that uses syntactic structures in contrast to deep learning approaches that encode whole sentences without taking into account syntactic features. We present an approach that uses combination of syntactic features and external word representations from FastText. The advantages of using FastText embeddings is the generation of better vector representations for rare words and FastText gives better results for words that are not within the dictionary. These attributes for vector representations give good results for morphologically rich languages. Most of the SRL approaches today are trained on resource rich languages. In this article we present novel neural architecture for SRL that is suitable for resource poor morphology rich languages. Experiments on hr500k corpus shows that our syntax-aware approach shows competitive results for argument classification. We present architecture for argument classification that is based on Bidirectional Long-Short Term Memory (Bi-LSTM) and Conditional Random Field (CRF) decoding for finding optimal sequence. Our approach showed results that are very close to benchmark results with F1 score of 72%.

State-of-the-art approaches to token labeling within text documents typically cast the problem either as a classification task, without using complex structural characteristics of the input, or as a sequential labeling task, carried out by a Conditional Random Field (CRF) classifier. Here we explore principled ways for structure to be brought to bear on the task. In line with recent trends in statistical learning of structured natural language input, we use a Support Vector Machine (SVM) classification framework deploying tree kernels. We then propose tree transformations and decorations, as a methodology for modeling complex linguistic phenomena in highly multi-dimensional feature spaces. We develop a general purpose tree engineering framework, which enables us to transcend the typically complex and laborious process of feature engineering. We build kernel based classifiers for two token labeling tasks: fine-grained event recognition, and lexical answer type detection in questions. For both, we show that in comparison with a corresponding linear kernel SVM, our method of using tree kernels improves recognition, thanks to appropriately engineering tree structures for use by the tree kernel. We also observe significant improvements when comparing with a CRF-based realization of structured prediction, itself performing at levels comparable to state-of-the-art.

This article presents a probabilistic scheme for detecting the interruption point (IP) in spontaneous speech based on inter-syllable boundary-based prosodic features. Because of the high error rate in spontaneous speech recognition, a combined acoustic model considering both syllable and subsyllable recognition units, is firstly used to determine the inter-syllable boundaries and output the recognition confidence of the input speech. Based on the finding that IPs always occur at inter-syllable boundaries, a probability distribution of the prosodic features at the current potential IP is estimated. The Conditional Random Field (CRF) model, which employs the clustered prosodic features of the current potential IP and its preceding and succeeding inter-syllable boundaries, is employed to output the IP likelihood measure. Finally, the confidence of the recognized speech, the probability distribution of the prosodic features and the CRF-based IP likelihood measure are integrated to determine the optimal IP sequence of the input spontaneous speech. In addition, pitch reset and lengthening are also applied to improve the IP detection performance. The Mandarin Conversional Dialogue Corpus is adopted for evaluation. Experimental results show that the proposed IP detection approach obtains 10.56% and 6.5% more effective results than the hidden Markov model and the Maximum Entropy model respectively under the same experimental conditions. Besides, the IP detection error rate can be further reduced by 9.15% using pitch reset and lengthening information. The experimental results confirm that the proposed model based on inter-syllable boundary-based prosodic features can effectively detect the interruption point in spontaneous Mandarin speech.

Weaponry equipment names belong to an important military naming entity that is difficult to identify because of features, such as complex components, miscellaneous, and scarce annotation corpus. Here, the automatic recognition of weaponry equipment names is specifically explored, a NER (Named Entity Recognition) algorithm is proposed based on BI-LSTM-CRF (Bi-directional Long Short Term Memory Conditional Random Field), thereby demonstrating the effectiveness of domain features in domain-specific entity recognition. Firstly, Chinese characters are represented by word embedding and input into the model. Then, the input feature vector sequence is processed by BI-LSTM (Bi-directional Long Short Term Memory) NN (Neural Network) to extract context semantic learning features. Finally, the learned features are connected to the linear CRF (Conditional Random Field), the NEs (Named Entities) in the equipment support field are labeled, and the NER results are obtained and output. The experimental results show that the accuracy of the NER algorithm based on the BI-LSTM-CRF model is 92.02%, the recall rate is 93.21%, and the F1 value reaches 93.88%. The effect of this model is better than the BI-LSTM NN model and LSTM-CRF (Long Short Term Memory Conditional Random Field) NN model. The proposed model provides some references for entity recognition in the field of equipment support.