Machine learning with naturally labeled data for identifying abbreviation definitions

Establishing structural or functional relationship between sequences, for instance to infer the structural class of an unannotated protein, is a key task in biological analysis. Protein sequences undergo complex transformations such as mutation, insertion and deletion during the evolutionary process and typically share low sequence similarity on the superfamily level, making the task for remote homology detection based on primary sequence only very challenging. Based on previous studies stating that knowledge based on only a subset of critical positions and the preferred symbols on such positions are sufficient for remote homology detection, we present a series of works, each enforcing different notion of sparsity, to recover such critical positions. We first start with a generative model and present the sparse profile hidden Markov models. Such generative approach recovers some critical patterns and motivates the need for discriminative learning. In our second study, we present a discriminative approach to recover such critical positions and the preferred symbols. In our third study, we address the issue of very few positive training examples, accompanied by a large number of negative training examples, which is typical in many remote homology detection task. Such issue motivates the need for semi-supervised learning. However, though containing abundant useful and critical information, large uncurated sequence databases also contain a lot of noise, which may compromise the quality of the classifiers. As a result, we present a systematic and biologically motivated framework for semi-supervised learning with large uncurated sequence databases. Combined with a very fast string kernel, our method not only realizes rapid and accurate remote homology detection and show state-of-the-art performance, but also recovers some critical patterns conserved in superfamilies.

This paper presents a novel method to retrieve images by learning the embedded visual concept from a set of given examples. Through a user's relevance feedback, the visual concept can be effectively learned to classify images which contain common visual entities. The learning process is started by providing a set of either positive or negative training examples and is then interactively adjusted according to the user's relevance feedback. In contrast to traditional methods, the proposed method utilizes a novel way to overcome the under-training problem which is frequently suffered in the learning process. Since no time-consuming optimization process is involved, the proposed method learns the visual concepts extremely fast. Therefore, the target concept can be learned on-line and is user-adaptable for effective retrieval of image contents. Experimental results are provided to prove the superiority of the proposed method.

A computational procedure to search for selective ligands for structurally related protein targets was developed and verified for serotonergic 5-HT7/5-HT1A receptor ligands. Starting from a set of compounds with annotated activity at both targets (grouped into four classes according to their activity: selective toward each target, not-selective and not-selective but active) and with an additional set of decoys (prepared using DUD methodology), the SVM (Support Vector Machines) models were constructed using a selective subset as positive examples and four remaining classes as negative training examples. Based on these four component models, the consensus classifier was then constructed using a data fusion approach. The combination of two approaches of data representation (molecular fingerprints vs. structural interaction fingerprints), different training set sizes and selection of the best SVM component models for consensus model generation, were evaluated to determine the optimal settings for the developed algorithm. The results showed that consensus models with molecular fingerprints, a larger training set and the selection of component models based on MCC maximization provided the best predictive performance.

The aim of this work is to localize a query photograph by finding other images depicting the same place in a large geotagged image database. This is a challenging task due to changes in viewpoint, imaging conditions and the large size of the image database. The contribution of this work is two-fold. First, we cast the place recognition problem as a classification task and use the available geotags to train a classifier for each location in the database in a similar manner to per-exemplar SVMs in object recognition. Second, as only few positive training examples are available for each location, we propose a new approach to calibrate all the per-location SVM classifiers using only the negative examples. The calibration we propose relies on a significance measure essentially equivalent to the p-values classically used in statistical hypothesis testing. Experiments are performed on a database of 25,000 geotagged street view images of Pittsburgh and demonstrate improved place recognition accuracy of the proposed approach over the previous work.

For hyper-rectangles in $$\mathbb{R}^{d}$$ Auer (1997) proved a PAC bound of $$O(\frac{1}{\varepsilon}(d+\log \frac{1}{\delta}))$$ , where $$\varepsilon$$ and $$\delta$$ are the accuracy and confidence parameters. It is still an open question whether one can obtain the same bound for intersection-closed concept classes of VC-dimension $$d$$ in general. We present a step towards a solution of this problem showing on one hand a new PAC bound of $$O(\frac{1}{\varepsilon}(d\log d + \log \frac{1}{\delta}))$$ for arbitrary intersection-closed concept classes, complementing the well-known bounds $$O(\frac{1}{\varepsilon}(\log \frac{1}{\delta}+d\log \frac{1}{\varepsilon}))$$ and $$O(\frac{d}{\varepsilon}\log \frac{1}{\delta})$$ of Blumer et al. and (1989) and Haussler, Littlestone and Warmuth (1994). Our bound is established using the closure algorithm, that generates as its hypothesis the intersection of all concepts that are consistent with the positive training examples. On the other hand, we show that many intersection-closed concept classes including e.g. maximum intersection-closed classes satisfy an additional combinatorial property that allows a proof of the optimal bound of $$O(\frac{1}{\varepsilon}(d+\log \frac{1}{\delta}))$$ . For such improved bounds the choice of the learning algorithm is crucial, as there are consistent learning algorithms that need $$\Omega(\frac{1}{\varepsilon}(d\log\frac{1}{\varepsilon} +\log\frac{1}{\delta}))$$ examples to learn some particular maximum intersection-closed concept classes.

An object detection method is developed based on combining top-down recognition with bottom-up image segmentation. There are three main steps in this method: a shape learning step, a hypothesis generation step and a verification step. In the shape learning step, training images are used to building codebook dictionary, and matching template is also constructed based on codebook dictionary. In the top-down hypothesis generation step, the Probability Shape Context (PSC) is used to generate a set of hypotheses of object locations. In the verification step, the hypotheses of object locations are segmented based on Partial Differential Equation (PDE). Based on segmented outlines and matching template, the inference is used to prune out false positives. Experimental results demonstrate that our proposed approach is able to accurately detect objects with only a few positive training examples.

Hierarchical learning strategy in semantic relation extraction

Density-aware generative algorithms learning from positive examples have verified high recall for visual object detection, but such generative methods suffer from excessive false positives which leads to low precision. Inspired by the recent success of detection-recognition pipeline with deep neural networks, this paper proposes a two-step framework by training a generative detector with positive samples first and then utilising a discriminative model to get rid of false positives in those detected bounding box candidates by the generative detector. Evidently, the discriminative model can be viewed as a post-processing step which improves the robustness by distinguishing true positives from false positives that confuse the generative detector. We exemplify the proposed approach on public ImageNet classes to demonstrate the significant improvement on precision while using only positive examples in training.

A common obstacle in effective learning of visual concept classifiers is the scarcity of positive training examples due to expensive labeling cost. This paper explores the sampling of weakly tagged web images for concept learning without human assistance. In particular, ontology knowledge is incorporated for semantic pooling of positive examples from ontologically neighboring concepts. This effectively widens the coverage of the positive samples with visually more diversified content, which is important for learning a good concept classifier. We experiment with two learning strategies: aggregate and incremental. The former strategy re-trains a new classifier by combining existing and newly collected examples, while the latter updates the existing model using the new samples incrementally. Extensive experiments on NUS-WIDE and VOC 2010 datasets show very encouraging results, even when comparing with classifiers learnt using expert labeled training examples.

BackgroundCognitive models of mood disorders emphasize a causal role of negative affective biases in depression. Computational work suggests that these biases may stem from a belief that negative events have a higher information content than positive events, resulting in preferential processing of and learning from negative outcomes. Learning biases therefore represent a promising target for therapeutic interventions. In this proof-of-concept study in healthy volunteers, we assessed the malleability of biased reinforcement learning using a novel cognitive training paradigm and concurrent transcranial direct current stimulation (tDCS).MethodsIn two studies, young healthy adults completed two sessions of negative (n = 20) or positive (n = 20) training designed to selectively increase learning from loss or win outcomes, respectively. During training active or sham tDCS was applied bilaterally to dorsolateral prefrontal cortex. Analyses tested for changes both in learning rates and win- and loss-driven behaviour. Potential positive/negative emotional transfer of win/loss learning was assessed by a facial emotion recognition task and mood questionnaires.ResultsNegative and positive training increased learning rates for losses and wins, respectively. With negative training, there was also a trend for win (but not loss) learning rates to decrease over successive task blocks. After negative training, there was evidence for near transfer in the form of an increase in loss-driven choices when participants performed a similar (untrained) task. There was no change in far transfer measures of emotional face processing or mood. tDCS had no effect on any aspect of behaviour.Discussion and ConclusionsNegative training induced a mild negative bias in healthy adults as reflected in loss-driven choice behaviour. Prefrontal tDCS had no effect. Further research is needed to assess if this training procedure can be adapted to enhance learning from positive outcomes and whether effects translate to affective disorders.

In this paper, a novel interactive medical image segmentation method called SMOPL is proposed. This method only needs marking some pixels on foreground region for segmentation. To do this, SMOPL characterize the inherent correlations among foreground and background pixels as Hilbert-Schmidt independence. By maximizing the independence and minimizing the smoothness of labels on instance neighbor graph simultaneously, SMOPL gets the sufficiently smooth confidences of both positive and negative classes in absence of negative training examples. Then a image segmentation can be obtained by assigning each pixel to the label for which the greatest confidence is calculated. Experiments on real-world medical images show that SMOPL is robust to get a high-quality segmentation with only positive label examples.

Graphical models are often used to represent and recognize activities. Purely unsupervised methods (such as HMMs) can be trained automatically but yield models whose internal structure - the nodes - are difficult to interpret semantically. Manually constructed networks typically have nodes corresponding to sub-events, but the programming and training of these networks is tedious and requires extensive domain expertise. In this paper, we propose a semi-supervised approach in which a manually structured, Propagation Network (a form of a DBN) is initialized from a small amount of fully annotated data, and then refined by an EM-based learning method in an unsupervised fashion. During node refinement (the M step) a boosting-based algorithm is employed to train the evidence detectors of individual nodes. Experiments on a variety of data types - vision and inertial measurements - in several tasks demonstrate the ability to learn from as little as one fully annotated example accompanied by a small number of positive but non-annotated training examples. The system is applied to both recognition and anomaly detection tasks.

We describe a method of pairwise 3D surface correspondence for the automated generation of landmarks on a set of examples from a class of shape. We show how the pairwise corresponder can be used in an extension of an existing framework for establishing dense correspondences between a set of training examples to build a 3D statistical model. The framework relies upon additional algorithms for the production of surface paths between vertices on a polyhedral mesh, and these are described. An example statistical model is shown for the left lateral ventricle of the brain.

An important problem in pattern recognition is that of pattern classification. The objective of classification is to determine a discriminant function which is consistent with the given training examples and performs reasonably well on an unlabeled test set of examples. The degree of performance of the classifier on the test examples, known as its generalization performance, is an important issue in the design of the classifier. It has been established that a good generalization performance can be achieved by providing the learner with a sufficiently large number of discriminative training examples. However, in many domains, it is infeasible or expensive to obtain a sufficiently large training set. Various mechanisms have been proposed in literature to combat this problem. Active Learning techniques (Angluin, 1998; Seung, Opper, & Sompolinsky, 1992) reduce the number of training examples required by carefully choosing discriminative training examples. Bootstrapping (Efron, 1979; Hamamoto, Uchimura & Tomita, 1997) and other pattern synthesis techniques generate a synthetic training set from the given training set. We present some of these techniques and propose some general mechanisms for pattern synthesis. Request access from your librarian to read this chapter's full text.

Effects of Interpretation Training on Hostile Attribution Bias and Reactivity to Interpersonal Insult