ViMMRC 2.0 — Enhancing Machine Reading Comprehension on Vietnamese Literature Text

Recent work has shown that predicate-argument label representations from semantic role labeling (SRL) can be concatenated with BERT representations to improve natural language understanding tasks such as natural language inference (NLI) and reading comprehension. Two natural questions that arise are whether infusing SRL representations with BERT 1) improves model performance and 2) increases the model’s linguistic awareness. This paper aims at answering both questions with a case study on the NLI task. We start by analyzing whether and how infusing SRL information helps BERT learn linguistic knowledge. We compare model performance on two benchmark datasets, SNLI and MNLI. We also conduct in-depth analysis on two probing datasets, Breaking NLI and HANS, which contain abundant examples where SRL information is expected to be helpful. We found that combining SRL representations with BERT representations does outperform BERT-only representations in general, with better awareness in lexical meaning and world knowledge but not in logic knowledge. We also found that infusing SRL information via predicate-wise concatenation with BERT word representations followed by an interaction layer is more effective than sentence-wise concatenation.

Semantic textual similarity(STS) and natural language inference(NLI) are important tasks in natural language processing(NLP) such as information retrieval, text classification, subject extraction, text summarization, machine translation and plagiarism detection. Lack of appropriate datasets in the Persian language is a major obstacle to progress in this area. Therefore, in this paper, we present FarSick, a new dataset for STS and NLI tasks in the Persian language. FarSick is the first relatively large-scale STS dataset for the Persian language. It includes 9804 pairs of Persian sentences with labels for similarity and inference for each pair of sentences. This dataset is collected by translating and editing the sentences of SICK dataset. We also measured the performance of traditional, statistical and deep learning models on it, e.g. transformers, Convolution Neural Networks, Bidirectional LSTMs, weighted average of word vectors, etc. We used different pre-trained embeddings, word2vec, glove, fastText and Bert sentence transformer. We used accuracy metric to test NLI tasks and Pearson metric to test STS tasks. The dataset is available at https://github.com/ZahraGhasemi-AI/FarSick.

The present study aimed to assess the effect of age and gender on second-grade children’s reading performance. Two hundred and eighty-seven children aged 7.1–8.2 years were divided into two age subgroups (the younger, 85–91 months and the older, 92–98 months) and were examined in reading accuracy, fluency, and comprehension tasks. Results showed a significant effect of age in reading performance, with the older children having better scores than younger ones for reading fluency, reading comprehension, and the total reading performance. Gender was not found to play an important role in reading performance. The findings are discussed on the ground of functional brain development and the different rates of cerebral maturation.

Component skills and discourse frameworks of reading have identified characteristics of readers and texts that influence comprehension. However, these 2 frameworks have not previously been integrated in a comprehensive and systematic way to explain performance on any standardized assessment of reading comprehension that is in widespread use across a broad developmental spectrum. The current study undertook such an integration using the reading comprehension subtest of the Gates-McGinitie Reading Tests, investigating characteristics of the test’s reading passages and comprehension questions and the component reading skills of students in middle and high school (n = 1,190) in order to determine the extent to which each of these dimensions affect test performance, both on average and through interaction with one another. The component skills of reading included word reading, reading fluency, vocabulary, background knowledge, and working memory. Test questions were coded for their processing demands, specifically the need to recall information from test and to form text-based inferences, while text passages were measured on word frequency, sentence length, text cohesion, and genre. We found that better vocabulary and background knowledge were the most important reader characteristics in accounting for reading comprehension. The processing demands of test questions were not highly predictive of item difficulty. Rather, genre was the most important passage feature in explaining item difficulty. Reader-test interactions were present, but were not pronounced. We discuss the importance of these findings as they relate to the measurement of comprehension and the integration of component skills and text-discourse models of reading. (PsycINFO Database Record (c) 2016 APA, all rights reserved)

Natural Language Inference (NLI) is a crucial task in natural language processing that involves determining the relationship between two sentences, typically referred to as the premise and the hypothesis. However, traditional NLI models solely rely on the semantic information inherent in independent sentences and lack relevant situational visual information, which can hinder a complete understanding of the intended meaning of the sentences due to the ambiguity and vagueness of language. To address this challenge, we propose an innovative ScenaFuse adapter that simultaneously integrates large-scale pre-trained linguistic knowledge and relevant visual information for NLI tasks. Specifically, we first design an image-sentence interaction module to incorporate visuals into the attention mechanism of the pre-trained model, allowing the two modalities to interact comprehensively. Furthermore, we introduce an image-sentence fusion module that can adaptively integrate visual information from images and semantic information from sentences. By incorporating relevant visual information and leveraging linguistic knowledge, our approach bridges the gap between language and vision, leading to improved understanding and inference capabilities in NLI tasks. Extensive benchmark experiments demonstrate that our proposed ScenaFuse, a scenario-guided approach, consistently boosts NLI performance.

Background: One of the documented design features of aphasia-friendly written information is the inclusion of graphics. People with aphasia have identified a preference for the inclusion of photographic illustrations in printed education materials (PEMs); however this preference contrasts with research suggesting that line drawings may be a more effective graphic type. Few studies have explored how graphics affect reading comprehension for people with aphasia. Aims: The aims of this research were (1) to determine if black and white line drawings and colour photographs affect the reading comprehension of people with and without aphasia, and (2) to explore perceptions of graphic helpfulness and preferences for including line drawings and photographic illustrations in PEMs. Methods & Procedures: A total of 22 people with aphasia and 15 of their significant others completed a purposefully developed cloze reading comprehension task, with multiple-choice response options. Response options contained (1) no illustrations, (2) black and white line drawings, or (3) colour photographs. The reading comprehension task was timed. The impact of no illustrations, line drawings, and photographs on (a) reading comprehension (measured in number of paragraphs correctly completed), and (b) response times (measured in seconds) was examined. Participants also completed a face-to-face graphic preference survey. Outcomes & Results: There were no significant differences on the reading comprehension task for participants with aphasia and those without, between (a) the number of paragraphs correctly completed in the set with no graphics, line drawings, and photographs, (b) the time taken to complete paragraphs in each of the three sets, and (c) the mean time taken to correctly complete paragraphs in each of the three sets. In contrast, the majority of participants perceived that pictures help understanding and make reading quicker. Significantly more participants with aphasia, compared to participants without, reported that they needed pictures to understand writing, and all participants with aphasia preferred health information to contain graphics. Several participants did not have a preference for line drawings or photographs in PEMs. Of those who did have a preference, more participants preferred written health information to contain photographs. Conclusions: Both participants with and without aphasia perceived that graphics in PEMs were helpful, however significantly more participants with aphasia reported that graphics were needed to understand text. The finding that line drawings and colour photographs did not significantly assist comprehension should not discount other benefits graphics may have in PEMs for people with aphasia.

Literacy demands have changed over the years and for success in society it is necessary to handle a wide range of texts and written information. The school has been criticized for not giving their pupils the necessary abilities to handle the kind of information they are faced with in society. One important dimension of literacy is reading comprehension, but even though much written information has the form of tables, drawings, graphs, etc. such presentations are most often accompanied by written text. This chapter focuses the comprehension of different kinds of written information, and data from different tasks are evaluated in light of the simple view of reading. A total of 132 grade 6 readers were given four reading comprehension tasks concurrently with a decoding task and a listening comprehension task. It was found that the sum of decoding and listening comprehension accounted for a larger part of the variance in all the reading comprehension tasks than the product of decoding and listening comprehension. The pupils' results on a naming task and morphological tasks from preschool accounted for significant parts of the variance in the comprehension of both plain text and text combined with tables and graphs over and above the concurrent decoding and listening comprehension results. Speed of orthographic identification in 2nd grade accounted for an additional, significant part of the variance in the plain text reading tasks. These results show that processing speed and linguistic knowledge, such as morphological knowledge, are important contributors to the comprehension of different kinds of written information. Even if speed of orthographic identification is especially important for comprehending plain texts, a broad linguistic and cognitive perspective seems to be important when preparing pupils to comprehend different kinds of written material.

Executive functions have both direct and indirect effects on reading comprehension. Previous studies have shown that people with hearing loss (HL) perform poorly on executive functions and reading comprehension tasks. This scoping review explored the current state of the literature relating executive function and reading comprehension in children and adolescents with cochlear implants. Searches were carried out in five databases to identify empirical studies related to executive functions, reading comprehension, and children and/or adolescents with cochlear implants. All results were analysed by peer-reviewers. A systematic search only found four studies in the field. We carried out our narrative review of these papers, with a focus on the mediating role of oral language on the relationship between executive functions and reading comprehension.

We examined the longitudinal contribution of awareness of inflections and derivations to reading comprehension in Arabic, a morphologically rich language, among 734 second graders. Morphological awareness, phonological awareness, word decoding and reading comprehension tasks were delivered at the beginning and at the end of the school year. Results indicated that readers improved in morphological awareness and reading comprehension over the course of the year. More importantly, in general, morphological awareness at the beginning of the year predicted reading comprehension at the end of the year. Moreover, inflectional awareness at the beginning of the year predicted reading comprehension at the end of the year among readers with low levels of morphological awareness, whereas both inflectional and derivational awareness predicted reading comprehension in high morphological awareness readers, beyond the contribution of word decoding. The results highlight the importance of testing awareness of inflections and derivations separately, and the differential role of inflectional and derivational awareness to Arabic reading comprehension in young readers with different levels of morphological awareness.

Assessing reading and online research comprehension: Do difficulties in attention and executive function matter?

Anomic Alexia of Kanji in a Patient with Anomic Aphasia

This study investigates the relationship of attitudes and of cultural background of 126 eighth-grade Arab students both in Israel and in Canada to their reading comprehension of majority and minority group cultural stories (i.e., Jewish and Arab cultural stories in Israel; Western and Arab cultural stories in Canada). Results showed that in the Israeli-Arab social context when students were given stories that reflected their cultural background, they scored higher on tasks of reading comprehension than when stories were culturally unfamiliar, regardless of the language of the text. However, in the Canadian context, the Arab students scored higher whenever they read stories in English, regardless of the cultural content of these stories. Furthermore, the results of the attitudes questionnaires showed that in Israel the motivation of Arab students in learning Hebrew was primarily instrumental rather than integrative. In the Canadian social context, Arab male students also showed only an instrumental motivation in learning English; however, it is noteworthy that in the Canadian context, Arab female students indicated a strong integrative motivation in learning English. The findings of this study indicate that the intersection between education and society cannot be ignored. In terms of language education, it is essential to develop appropriate pedagogies that respond to the diversity of the social contexts in which the learners are situated.

Objective This paper examines the impact of the level of educational opportunities (LEO) on students’ reading skills test performance. Method A total of 609 students in the first three years of secondary school in Argentina were evaluated on reading efficiency and comprehension tasks across three text types: narrative, expository, and argumentative. A two-way ANOVA was conducted with LEO (low, medium, and high) and year level (first, second, and third year) as independent variables for each of these assessments. We hypothesized that scores would improve as year level increased and that LEO would significantly affect all variables. Results Results show that year level influenced performance across all measures. LEO had an impact on reading efficiency and comprehension of expository and argumentative texts, but not narrative text comprehension. Furthermore, an interaction effect was found in argumentative text comprehension. Conclusions These findings provide empirical evidence that LEO significantly influences various reading and text comprehension tasks, highlighting the importance of improving school conditions as a key factor in public policy aimed at enhancing essential learning outcomes.

Natural Language Inference (NLI) datasets contain examples with highly ambiguous labels. While many research works do not pay much attention to this fact, several recent efforts have been made to acknowledge and embrace the existence of ambiguity, such as UNLI and ChaosNLI. In this paper, we explore the option of training directly on the estimated label distribution of the annotators in the NLI task, using a learning loss based on this ambiguity distribution instead of the gold-labels. We prepare AmbiNLI, a trial dataset obtained from readily available sources, and show it is possible to reduce ChaosNLI divergence scores when finetuning on this data, a promising first step towards learning how to capture linguistic ambiguity. Additionally, we show that training on the same amount of data but targeting the ambiguity distribution instead of gold-labels can result in models that achieve higher performance and learn better representations for downstream tasks.

Reading disorders, including reading comprehension disorders, are among the most common referrals for evaluation in schools. If that evaluation involves individually administered tests of reading, the examiner is faced with selecting at least one reading comprehension subtest that is inherently associated with specific task demands, such as a cloze procedure or a story retell procedure. This study explored the correlates of performance of these two types of reading comprehension measures in lower grades (i.e., grades 1–5) and upper grades (i.e., grades 6–12). Results revealed a moderate correlation between these two tasks and evidence that students with reading disorders may perform poorer on cloze reading comprehension measures than story retell measures in both grade groupings. Regression analyses revealed that variance in each reading comprehension task is associated with a unique grouping of predictor variables that are associated with the Big Five of Reading and short-term/working memory. Discussion focuses on the implications of these findings for the evaluation of students with suspected disabilities and research.