Bridging-Inferences and Enthymemes

We probe pre-trained transformer language models for bridging inference. We first investigate individual attention heads in BERT and observe that attention heads at higher layers prominently focus on bridging relations in-comparison with the lower and middle layers, also, few specific attention heads concentrate consistently on bridging. More importantly, we consider language models as a whole in our second approach where bridging anaphora resolution is formulated as a masked token prediction task (Of-Cloze test). Our formulation produces optimistic results without any fine-tuning, which indicates that pre-trained language models substantially capture bridging inference. Our further investigation shows that the distance between anaphor-antecedent and the context provided to language models play an important role in the inference.

This study focuses on first year biology students at Stockholm University who are assumed by the Swedish university system to be able to read textbooks in English. These students are compared with first year biology students at two British universities, all of whom were first language (L1) readers of English. Several components of reading comprehension are compared. The Swedish readers are subdivided on the basis of a reading comprehension test into two subgroups, referred to as the ‘very’ advanced and the ‘less’ advanced second language (L2) groups. Other measures of reading skill are then used to compare the two Swedish groups with each other and with the L1 group. The results indicate that the very advanced Swedish readers of English are fully comparable to the L1 readers on reading comprehension and inferencing tasks. However, these very advanced L2 students were nonetheless somewhat weaker on vocabulary, recall, and word recognition (lexical decision) tasks when compared to the L1 students. It is speculated that the L2 readers generally use inferencing, in the form of general world knowledge and bridging inferences, as a means of compensating for lack of automaticity in low level processing. The less advanced Swedish readers, who comprise the majority of the L2 students, were considerably weaker than the very advanced students on all tests, suggesting that these students may in fact be considerably disadvantaged in having to read course material (in this case biology) in English. A coefficient of variation was calculated for the word recognition speed test and the sentence coherence recognition speed test as an indicator of automaticity for the L1 group and the L2 group as a whole. For word recognition there was a clear difference, but for sentence coherence the difference was less apparent. A possible explanation is that some of the sentence coherence test items were contrary to expectation and forced the L1 subjects to slow down and consciously work out whether the sentence was coherent or not. The general conclusions are (1) that the majority of Swedish students do not meet the English reading expectations of the university system, and (2) a minority of the Swedish students, although they had lower scores on vocabulary and recall tasks and less automaticity on the response-time tasks than the L1 readers, seemed to compensate for these weaknesses with inferencing skills, allowing them to achieve comparable reading comprehension product to the L1 readers.

Students in Psychology, English, and Natural Science were invited to submit questions for information deemed by them pertinent to success in a course. A 13-category classification of the 1030 items collected from 194 students showed dominance of personal and teacher-related questions. Mean number of questions for upper classmen were consistently lower than those for lower classmen, this being interpreted as a normative and developmental tendency. Types of questions were restricted to cultural norms that centered on personal traits, interests, attitudes, opinions, and work of the target person, rather than on interpersonal relationships, morality, sex, and personal concerns. Analysis of class-size effects indicated that students attending a large class asked significantly more questions than those attending a small class in one of the four categories assessed, grading practices. Lower classmen tended to ask more questions about acceptable classroom behavior than upper classmen.

We introduce Episodic Memory QA, the task of answering personal user questions grounded on memory graph (MG), where episodic memories and related entity nodes are connected via relational edges. We create a new benchmark dataset first by generating synthetic memory graphs with simulated attributes, and by composing 100K QA pairs for the generated MG with bootstrapped scripts. To address the unique challenges for the proposed task, we propose Memory Graph Networks (MGN), a novel extension of memory networks to enable dynamic expansion of memory slots through graph traversals, thus able to answer queries in which contexts from multiple linked episodes and external knowledge are required. We then propose the Episodic Memory QA Net with multiple module networks to effectively handle various question types. Empirical results show improvement over the QA baselines in top-k answer prediction accuracy in the proposed task. The proposed model also generates a graph walk path and attention vectors for each predicted answer, providing a natural way to explain its QA reasoning.

The theory of Boolean functions, especially in respect to representing these functions in the disjunctive or conjunctive normal forms, is extended in this chapter onto the case of finite predicates. Finite predicates are decomposed by that into some binary units, which will correspond to components of Boolean vectors and matrices and are represented as combinations of these units. Further, the main concepts used for solving pattern recognition problems are defined, namely world model, data, and knowledge. The data presenting information about the existence of some objects with definite combinations of properties is considered, as well as the knowledge presenting information about the existence of regular relationships between attributes. These relationships prohibit some combinations of properties. In this way, the knowledge gives the information about the non-existence of objects with some definite (prohibited) combinations of attribute values. A special form of regularity representation, called implicative regularities, is introduced. Any implicative regularity generates an empty interval in the Boolean space of object descriptions, which do not contradict the data. The problem of plausibility evaluation of induced implicative regularities should be solved by that. The pattern recognition problem is solved by two steps. First, regularities are extracted from the database (inductive inference); second, the obtained knowledge is used for the object recognition (deductive inference).

BackgroundNext-generation sequencing technologies have profoundly impacted biology over recent years. Experimental protocols, such as photoactivatable ribonucleoside-enhanced cross-linking and immunoprecipitation (PAR-CLIP), which identifies protein–RNA interactions on a genome-wide scale, commonly employ deep sequencing. With PAR-CLIP, the incorporation of photoactivatable nucleosides into nascent transcripts leads to high rates of specific nucleotide conversions during reverse transcription. So far, the specific properties of PAR-CLIP-derived sequencing reads have not been assessed in depth.MethodsWe here compared PAR-CLIP sequencing reads to regular transcriptome sequencing reads (RNA-Seq) to identify distinctive properties that are relevant for reference-based read alignment of PAR-CLIP datasets. We developed a set of freely available tools for PAR-CLIP data analysis, called the PAR-CLIP analyzer suite (PARA-suite). The PARA-suite includes error model inference, PAR-CLIP read simulation based on PAR-CLIP specific properties, a full read alignment pipeline with a modified Burrows–Wheeler Aligner algorithm and CLIP read clustering for binding site detection.ResultsWe show that differences in the error profiles of PAR-CLIP reads relative to regular transcriptome sequencing reads (RNA-Seq) make a distinct processing advantageous. We examine the alignment accuracy of commonly applied read aligners on 10 simulated PAR-CLIP datasets using different parameter settings and identified the most accurate setup among those read aligners. We demonstrate the performance of the PARA-suite in conjunction with different binding site detection algorithms on several real PAR-CLIP and HITS-CLIP datasets. Our processing pipeline allowed the improvement of both alignment and binding site detection accuracy.AvailabilityThe PARA-suite toolkit and the PARA-suite aligner are available at https://github.com/akloetgen/PARA-suite and https://github.com/akloetgen/PARA-suite_aligner, respectively, under the GNU GPLv3 license.