Hierarchical learning strategy in semantic relation extraction

Achieving asymptotic and concurrent fault diagnosis in hydraulic system remains a challenging endeavor due to the inherent attributes of the hidden occurrence, simultaneous manifestation, coupling, and limited sample size. To address the above issues, this paper proposes a hierarchical multi-output fault detection and diagnosis framework, namely, HMDF, based on a hierarchical learning strategy to leverage an improved convolutional neural network (CNN) and support vector machine (SVM). Both a multi-channel CNN and a multi-branch CNN are employed to extract and downscale features collected by the sensors at diverse sampling frequencies first, and then, such features are subsequently subjected to classification using SVM. The hierarchical learning strategy enables the identification of different fault states, both at the component and the intra-component level. Additionally, a modified whale optimization algorithm is also utilized to optimize the classification process of SVM. Extensive experiments are conducted to test the proposed HMDF with the hydraulic system datasets. Results show that HMDF achieves a diagnostic accuracy of up to 98.9% for the dataset, surpassing traditional methods reliant on manual extraction of time–frequency features, and it also exhibits superior classification performances with a small sample size. The HMDF is expected to offer a generalized framework for the multi-output fault detection and diagnosis in hydraulic systems and other complex components.

Density-aware generative algorithms learning from positive examples have verified high recall for visual object detection, but such generative methods suffer from excessive false positives which leads to low precision. Inspired by the recent success of detection-recognition pipeline with deep neural networks, this paper proposes a two-step framework by training a generative detector with positive samples first and then utilising a discriminative model to get rid of false positives in those detected bounding box candidates by the generative detector. Evidently, the discriminative model can be viewed as a post-processing step which improves the robustness by distinguishing true positives from false positives that confuse the generative detector. We exemplify the proposed approach on public ImageNet classes to demonstrate the significant improvement on precision while using only positive examples in training.

A common obstacle in effective learning of visual concept classifiers is the scarcity of positive training examples due to expensive labeling cost. This paper explores the sampling of weakly tagged web images for concept learning without human assistance. In particular, ontology knowledge is incorporated for semantic pooling of positive examples from ontologically neighboring concepts. This effectively widens the coverage of the positive samples with visually more diversified content, which is important for learning a good concept classifier. We experiment with two learning strategies: aggregate and incremental. The former strategy re-trains a new classifier by combining existing and newly collected examples, while the latter updates the existing model using the new samples incrementally. Extensive experiments on NUS-WIDE and VOC 2010 datasets show very encouraging results, even when comparing with classifiers learnt using expert labeled training examples.

Establishing structural or functional relationship between sequences, for instance to infer the structural class of an unannotated protein, is a key task in biological analysis. Protein sequences undergo complex transformations such as mutation, insertion and deletion during the evolutionary process and typically share low sequence similarity on the superfamily level, making the task for remote homology detection based on primary sequence only very challenging. Based on previous studies stating that knowledge based on only a subset of critical positions and the preferred symbols on such positions are sufficient for remote homology detection, we present a series of works, each enforcing different notion of sparsity, to recover such critical positions. We first start with a generative model and present the sparse profile hidden Markov models. Such generative approach recovers some critical patterns and motivates the need for discriminative learning. In our second study, we present a discriminative approach to recover such critical positions and the preferred symbols. In our third study, we address the issue of very few positive training examples, accompanied by a large number of negative training examples, which is typical in many remote homology detection task. Such issue motivates the need for semi-supervised learning. However, though containing abundant useful and critical information, large uncurated sequence databases also contain a lot of noise, which may compromise the quality of the classifiers. As a result, we present a systematic and biologically motivated framework for semi-supervised learning with large uncurated sequence databases. Combined with a very fast string kernel, our method not only realizes rapid and accurate remote homology detection and show state-of-the-art performance, but also recovers some critical patterns conserved in superfamilies.

BackgroundThe rapid growth of biomedical literature requires accurate text analysis and text processing tools. Detecting abbreviations and identifying their definitions is an important component of such tools. Most existing approaches for the abbreviation definition identification task employ rule-based methods. While achieving high precision, rule-based methods are limited to the rules defined and fail to capture many uncommon definition patterns. Supervised learning techniques, which offer more flexibility in detecting abbreviation definitions, have also been applied to the problem. However, they require manually labeled training data.MethodsIn this work, we develop a machine learning algorithm for abbreviation definition identification in text which makes use of what we term naturally labeled data. Positive training examples are naturally occurring potential abbreviation-definition pairs in text. Negative training examples are generated by randomly mixing potential abbreviations with unrelated potential definitions. The machine learner is trained to distinguish between these two sets of examples. Then, the learned feature weights are used to identify the abbreviation full form. This approach does not require manually labeled training data.ResultsWe evaluate the performance of our algorithm on the Ab3P, BIOADI and Medstract corpora. Our system demonstrated results that compare favourably to the existing Ab3P and BIOADI systems. We achieve an F-measure of 91.36% on Ab3P corpus, and an F-measure of 87.13% on BIOADI corpus which are superior to the results reported by Ab3P and BIOADI systems. Moreover, we outperform these systems in terms of recall, which is one of our goals.

Addressed in this paper is the issue of semantic relationship extraction from semi-structured documents. Many research efforts have been made so far on the semantic information extraction. However, much of the previous work focuses on detecting `isolated' semantic information by making use of linguistic analysis or linkage information in web pages and limited research has been done on extracting semantic relationship from the semi-structured documents. In this paper, we propose a method for semantic relationship extraction by using the logical information in the semi-structured document (semi-structured document usually has various types of structure information, e.g. a semi-structured document may be hierarchical laid out). To the best of our knowledge, extracting semantic relationships by using logical information has not been investigated previously. A probabilistic approach has been proposed in the paper. Features used in the probabilistic model have been defined.

The aim of this work is to localize a query photograph by finding other images depicting the same place in a large geotagged image database. This is a challenging task due to changes in viewpoint, imaging conditions and the large size of the image database. The contribution of this work is two-fold. First, we cast the place recognition problem as a classification task and use the available geotags to train a classifier for each location in the database in a similar manner to per-exemplar SVMs in object recognition. Second, as only few positive training examples are available for each location, we propose a new approach to calibrate all the per-location SVM classifiers using only the negative examples. The calibration we propose relies on a significance measure essentially equivalent to the p-values classically used in statistical hypothesis testing. Experiments are performed on a database of 25,000 geotagged street view images of Pittsburgh and demonstrate improved place recognition accuracy of the proposed approach over the previous work.

For hyper-rectangles in $$\mathbb{R}^{d}$$ Auer (1997) proved a PAC bound of $$O(\frac{1}{\varepsilon}(d+\log \frac{1}{\delta}))$$ , where $$\varepsilon$$ and $$\delta$$ are the accuracy and confidence parameters. It is still an open question whether one can obtain the same bound for intersection-closed concept classes of VC-dimension $$d$$ in general. We present a step towards a solution of this problem showing on one hand a new PAC bound of $$O(\frac{1}{\varepsilon}(d\log d + \log \frac{1}{\delta}))$$ for arbitrary intersection-closed concept classes, complementing the well-known bounds $$O(\frac{1}{\varepsilon}(\log \frac{1}{\delta}+d\log \frac{1}{\varepsilon}))$$ and $$O(\frac{d}{\varepsilon}\log \frac{1}{\delta})$$ of Blumer et al. and (1989) and Haussler, Littlestone and Warmuth (1994). Our bound is established using the closure algorithm, that generates as its hypothesis the intersection of all concepts that are consistent with the positive training examples. On the other hand, we show that many intersection-closed concept classes including e.g. maximum intersection-closed classes satisfy an additional combinatorial property that allows a proof of the optimal bound of $$O(\frac{1}{\varepsilon}(d+\log \frac{1}{\delta}))$$ . For such improved bounds the choice of the learning algorithm is crucial, as there are consistent learning algorithms that need $$\Omega(\frac{1}{\varepsilon}(d\log\frac{1}{\varepsilon} +\log\frac{1}{\delta}))$$ examples to learn some particular maximum intersection-closed concept classes.

An object detection method is developed based on combining top-down recognition with bottom-up image segmentation. There are three main steps in this method: a shape learning step, a hypothesis generation step and a verification step. In the shape learning step, training images are used to building codebook dictionary, and matching template is also constructed based on codebook dictionary. In the top-down hypothesis generation step, the Probability Shape Context (PSC) is used to generate a set of hypotheses of object locations. In the verification step, the hypotheses of object locations are segmented based on Partial Differential Equation (PDE). Based on segmented outlines and matching template, the inference is used to prune out false positives. Experimental results demonstrate that our proposed approach is able to accurately detect objects with only a few positive training examples.

A large volume of unstructured data, especially text data, is generated and exchanged daily. Consequently, the importance of extracting patterns and discovering knowledge from textual data is significantly increasing. As the task of automatically recognizing the relations between two or more entities, semantic relation extraction has a prominent role in the exploitation of raw text. This article surveys different approaches and types of relation extraction in English and the most prominent proposed methods in Persian. We also introduce, analyze, and compare the most important datasets available for relation extraction in Persian and English. Furthermore, traditional and emerging evaluation metrics for supervised, semi-supervised, and unsupervised methods are described, along with pointers to commonly used performance evaluation datasets. Finally, we briefly describe challenges in extracting relationships in Persian and English and dataset creation challenges.

Aggregate Reinforcement Learning for multi-agent territory division: The Hide-and-Seek game

This paper presents a novel method to retrieve images by learning the embedded visual concept from a set of given examples. Through a user's relevance feedback, the visual concept can be effectively learned to classify images which contain common visual entities. The learning process is started by providing a set of either positive or negative training examples and is then interactively adjusted according to the user's relevance feedback. In contrast to traditional methods, the proposed method utilizes a novel way to overcome the under-training problem which is frequently suffered in the learning process. Since no time-consuming optimization process is involved, the proposed method learns the visual concepts extremely fast. Therefore, the target concept can be learned on-line and is user-adaptable for effective retrieval of image contents. Experimental results are provided to prove the superiority of the proposed method.

Graphical models are often used to represent and recognize activities. Purely unsupervised methods (such as HMMs) can be trained automatically but yield models whose internal structure - the nodes - are difficult to interpret semantically. Manually constructed networks typically have nodes corresponding to sub-events, but the programming and training of these networks is tedious and requires extensive domain expertise. In this paper, we propose a semi-supervised approach in which a manually structured, Propagation Network (a form of a DBN) is initialized from a small amount of fully annotated data, and then refined by an EM-based learning method in an unsupervised fashion. During node refinement (the M step) a boosting-based algorithm is employed to train the evidence detectors of individual nodes. Experiments on a variety of data types - vision and inertial measurements - in several tasks demonstrate the ability to learn from as little as one fully annotated example accompanied by a small number of positive but non-annotated training examples. The system is applied to both recognition and anomaly detection tasks.

In this paper, we proposed a novel approach based on topic ontology for tag recommendation. The proposed approach intelligently generates tag suggestions to blogs. In this approach, we construct topic ontology through enriching the set of categories in existing small ontology called as Open Directory Project. To construct topic ontology, a set of topics and their associated semantic relationships is identified automatically from the corpus-based external knowledge resources such as Wikipedia and WordNet. The construction relies on two folds such as concept acquisition and semantic relation extraction. In the first fold, a topic-mapping algorithm is developed to acquire the concepts from the semantic of Wikipedia. A semantic similarity-clustering algorithm is used to compute the semantic similarity measure to group the set of similar concepts. The second is the semantic relation extraction algorithm, which derives associated semantic relations between the set of extracted topics from the lexical patterns between synsets in WordNet. A suitable software prototype is created to implement the topic ontology construction process. A Jena API framework is used to organize the set of extracted semantic concepts and their corresponding relationship in the form of knowledgeable representation of Web ontology language. Thus, Protege tool provides the platform to visualize the automatically constructed topic ontology successfully. Using the constructed topic ontology, we can generate and suggest the most suitable tags for the new resource to users. The applicability of topic ontology with a spreading activation algorithm supports efficient recommendation in practice that can recommend the most popular tags for a specific resource. The spreading activation algorithm can assign the interest scores to the existing extracted blog content and tags. The weight of the tags is computed based on the activation score determined from the similarity between the topics in constructed topic ontology and content of the existing blogs. High-quality tags that has the highest activation score is recommended to the users. Finally, we conducted experimental evaluation of our tag recommendation approach using a large set of real-world data sets. Our experimental results explore and compare the capabilities of our proposed topic ontology with the spreading activation tag recommendation approach with respect to the existing AutoTag mechanism. And also discuss about the improvement in precision and recall of recommended tags on the data sets of Delicious and BibSonomy. The experiment shows that tag recommendation using topic ontology results in the folksonomy enrichment. Thus, we report the results of an experiment mean to improve the performance of the tag recommendation approach and its quality. © 2012 Wiley Periodicals, Inc.

Semantic relation extraction has been a known branch in Natural language processing allowing the identification of relations between words. Knowledge-based approaches including wordnet and pattern matching approaches such as Hearst for hypernyms are used among many others. The search for relations is limited to sentences which avert the rest of the text narrowing the semantic relation extraction. In this paper, we introduce the use of context for improving the extraction using global information from the text and domain ontology.