The problem of computing the most probable tree in data-oriented parsing and stochastic tree grammars

This article presents an analysis of experiments with statistical and neural parsing techniques for Urdu, a widely spoken South Asian language. We demonstrate state of the art constituency parsing results for an Urdu treebank. Urdu is a morphologically rich and is characterized by free word order. Language representation (e.g. input type, lemmatization, word clusters), part of speech tag set, phrase labels and the size of a training corpus are crucial for parsing such languages. In this article, probabilistic context-free grammars, data-oriented parsing, and recursive neural network based models have been experimented with several linguistic features which show improvements in the parsing results. Features include syntactic sub-categorization of POS tags, empirically learned horizontal and vertical markovizations and lexical head words. These features enable dependency information for case markers and add phrasal and lexical context to the parse trees. The data-oriented parsing and recursive neural network model give an f-score of 87.1 by considering gold POS tags in the test set, on textual input, they show a performance with f-scores of 83.4 and 84.2, respectively. To overcome the issue of data sparsity due to the morphological richness, lemmatization and unsupervised word clustering have been performed. A treebank should cover most probable word orders of the language so that models can learn various orders accurately. To analyze the order coverage of the treebank and learning capability of different parsers, a test set has been prepared conditioning different word orders. This test set is evaluated with the best performing parsing models and with gold POS tags, f-scores are above 90 and on textual input, the average f-score is 87.6.

This paper presents a new algorithm in computer vision for the recognition of hand gestures. In the proposed system, Kinect sensor is used to track and segment hand in the clutter background and feature extracted by finger and an angle between them. Classify the hand posture using multi-class support vector machine. The hand gesture is recognised by stochastic context free grammar (SCFG). Stochastic context free grammar uses syntactic structure analysis and by this method, recognises hand gestures by set of production rules which consists of a combination of hand postures. The proposed algorithm is able to recognise various hand postures in real time with more than 97% accuracy.

Computing the relative entropy between regular tree languages

A New Method for Predicting Protein Secondary Structures Based on Stochastic Tree Grammars

Given a sample from an unknown probability distribution over strings, there exist algorithms for inferring the structure and parameters of stochastic grammatical representations of the unknown distribution, i.e. string grammars. Despite the fact that research on grammatical representations of sets of graphs has been conducted since the late 1960’s, almost no work has considered the possibility of stochastic graph grammars and no algorithms exist for inferring stochastic graph grammars from data. This paper presents PEGG, an algorithm for estimating the parameters of stochastic context-free graph grammars given a sample from an unknown probability distribution over graphs. It is established formally that for a certain class of graph grammars PEGG finds parameter estimates in polynomial time that maximize the likelihood of the data, and preliminary empirical results demonstrate that the algorithm performs well in practice.KeywordsGraph TransformationGraph GrammarMaximum Likelihood ParameterStart SymbolEstimate Maximum Likelihood ParameterThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A syntactic approach to texture analysis

Structure detection and segmentation of documents using 2D stochastic context-free grammars

A data-oriented parsing or DOP model for statistical parsing associates fragments of linguistic representations with numerical weights, where these weights are estimated by normalizing the empirical frequency of each fragment in a training corpus (see Bod [1998] and references cited therein). This note observes that this estimation method is biased and inconsistent that is, the estimated distribution does not in general converge on the true distribution as the size of the training corpus increases.

The aim of syntactic tracking is to classify spatio-temporal patterns of a target's motion using natural language processing models. In this paper, we generalize earlier work by considering a constrained stochastic context free grammar (CSCFG) for modeling patterns confined to a roadmap. The constrained grammar facilitates modeling specific directions and road names in a roadmap. We present a novel particle filtering algorithm that exploits the CSCFG model for estimating the target's patterns. This meta-level algorithm operates in conjunction with a base-level tracking algorithm. Extensive numerical results using simulated ground moving target indicator (GMTI) radar measurements show substantial improvement in target tracking accuracy.

Parsing is a tool for understanding natural grammar patterns. The problem of structural ambiguity in identifying sentence patterns often occurs in parsing. Syntactic parsing is one approach to solving structural ambiguity problems using the Probabilistic Context-Free Grammar (PCFG) and Viterbi-Cocke Younger Kasami (Viterbi-CYK) methods. Meanwhile, a large number of Indonesian language resources are needed as machine knowledge to parse. This research build a parsing of Indonesian sentence patterns with Indonesian Tagged corpus resource then solve the ambiguity problem of Indonesian sentence pattern parsing using PCFG and Viterbi-CYK algorithms. The corpus data is processed to obtain grammar rules using the PCFG algorithm. Then, the sentence on the corpus is processed by the PCFG rule that generated and uses the Viterbi-CYK algorithm to get the parse tree taken based on the highest probability value. The results of the research produced an average value of similarity production rules which the highest values is 92.95%. This shows that the Indonesian parsing successfully parses Indonesian sentence and can solve the problem of structural ambiguity in the parsing of Indonesian sentence patterns.

Although stochastic context-free grammars (SCFGs) appear promising for recognition of radar emitters, and for estimation of their level of threat in radar electronic support (ES) systems, well-known techniques for learning their production rule probabilities are computationally demanding, and cannot efficiently reflect changes in operational environments. Some techniques have been proposed for fast learning of SCFGs probabilities, yet, of those, only the HOLA technique can perform learning incrementally. In this paper, two incremental versions of the graphical EM (gEM) technique are proposed. The incremental gEM (igEM) and on-line incremental gEM (oigEM) allow for adapting production rule probabilities from new data, without having to retrain from the start on all accumulated training data. These new techniques are compared to HOLA using radar signal data. An experimental protocol has been defined such that the impact on performance of factors like the size of new data blocks for incremental learning, and the level of ambiguity of MFR grammars, may be observed. Results indicate that, contrary to HOLA, incremental learning of training data blocks with igEM and oigEM provides the same level of accuracy as learning from all cumulative data from scratch, even for small data blocks. As expected, incremental learning significantly reduces the overall time and memory complexities. Finally, it appears that while the computational complexity and memory requirements of igEM and oigEM may be greater than that of HOLA, they both provide a higher level of accuracy.

The current and future business processes would be service oriented. The required service is selected using discovery process. The discovery process uses in-memory parse tree construction or callback based document parsing. Existing service discovery systems stores both functional attributed and QoS attributed in tree data structures and they are time consuming, space consuming, requires frequent parse tree construction in various location, difficult to replicate and distribute parse tree and replicated parse trees are inconsistent. This paper proposes an efficient methodology to improve the performance of in memory parse tree discovery process over extended web service architecture. The system uses two parallel discovery algorithms, Synchronous and Asynchronous algorithms for service discovery, both algorithms uses the tree and indexed data structure for service discovery and the later also uses a bitmap table to verify functional requirements. The services are selected and ranked based on user request using Coefficient of Variance. The proposed algorithms are detailed in this paper. The experimental results are conducted on benchmark dataset are discussed. The proposed system ensures availability, consistency, effective space utilization, eases replication and distribution and improves response time for service discovery by 30%.

Structured prediction algorithms—used when applying machine learning to tasks like natural language parsing and image understanding—present some opportunities for fine-grained parallelism, but also have problem-specific serial dependencies. Most implementations exploit only simple opportunities such as parallel BLAS, or embarrassing parallelism over input examples. In this work we explore an orthogonal direction: using the fact that these algorithms can be described as specialized forward-chaining theorem provers [1], [2], and implementing fine-grained parallelization of the forward-chaining mechanism. We study context-free parsing as a simple canonical example, but the approach is more general.

Conformational entropy for atomic-level, three dimensional biomolecules is known experimentally to play an important role in protein-ligand discrimination, yet reliable computation of entropy remains a difficult problem. Here we describe the first two accurate and efficient algorithms to compute the conformational entropy for RNA secondary structures, with respect to the Turner energy model, where free energy parameters are determined from UV absorption experiments. An algorithm to compute the derivational entropy for RNA secondary structures had previously been introduced, using stochastic context free grammars (SCFGs). However, the numerical value of derivational entropy depends heavily on the chosen context free grammar and on the training set used to estimate rule probabilities. Using data from the Rfam database, we determine that both of our thermodynamic methods, which agree in numerical value, are substantially faster than the SCFG method. Thermodynamic structural entropy is much smaller than derivational entropy, and the correlation between length-normalized thermodynamic entropy and derivational entropy is moderately weak to poor. In applications, we plot the structural entropy as a function of temperature for known thermoswitches, such as the repression of heat shock gene expression (ROSE) element, we determine that the correlation between hammerhead ribozyme cleavage activity and total free energy is improved by including an additional free energy term arising from conformational entropy, and we plot the structural entropy of windows of the HIV-1 genome. Our software RNAentropy can compute structural entropy for any user-specified temperature, and supports both the Turner’99 and Turner’04 energy parameters. It follows that RNAentropy is state-of-the-art software to compute RNA secondary structure conformational entropy. Source code is available at https://github.com/clotelab/RNAentropy/; a full web server is available at http://bioinformatics.bc.edu/clotelab/RNAentropy, including source code and ancillary programs.

In this chapter we present a method for learning a compositional model in a minimax entropy framework for modeling object categories with large intra-class variance. The model we learn incorporates the flexibility of a stochastic context free grammar (SCFG) to account for the variation in object structure with the neighborhood constraints of a Markov random field (MRF) to enforce spatial context. We learn the model through a generalized minimax entropy framework that accounts for the dynamic structure of the hierarchical model. We first learn the SCFG parameters using the frequencies of object parts, then pursue spatial relations in order of greatest information gain. The learned model can generalize from a small set of training samples (n < 100) to generate a combinatorially large number of novel instances using stochastic sampling. To verify our learning method and model performance, we present plots of KL divergence minimization as the algorithm proceeds, and show that samples from the model become more realistic as more spatial relations are added. We also show the model accurately predicting missing or undetected parts for top-down recognition along with preliminary results showing that the model can learn a large space of category appearances from a very small (n < 15) number of training samples. This process is similar to “recognition-by-components”, a theory that postulates that biological vision systems recognize objects as composed from a dictionary of commonly appearing 3D structures. Finally, we discuss a compositional boosting algorithm for inference and show examples using it for object recognition. This article is a chapter from the book Object Categorization: Computer and Human Vision Perspectives, edited by Sven Dickinson, Ales Leonardis, Bernt Schiele, and Michael J. Tarr (Cambridge University Press). University of California Los Angeles, Los Angeles, CA. Lotus Hill Research Institute, EZhou, China.