Research on the Application of Machine Reading Comprehension in Adaptive Evaluation System

Understanding trainers’ perceptions of innovation is the key to successful adoption of technology in learning. The purpose of this paper was to examine the level of trainers’ perception on Information Communication Technology (ICT) integration into teaching and learning of engineering courses in National polytechnics in Kenya. This study used mixed method research. The target population in this study was 140 engineering staff and the trainees, Heads of departments and Principals of Kisumu and Eldoret Nationa l Polytechnic and Kenya Technical Trainers College. Primary data was used, which was collected using structured questionnaires and interview schedules. Simple random sampling was used to identify the respondents from the relevant departments to participate in this study, while purposive sampling was used to select the principals to be interviewed. Reliability and validity of data collection instruments were ascertained through pilot testing. Quantitative data was analyzed using descriptive and inferential statistics, while qualitative data generated from the interview schedules was organized through content scrutiny. The results indicated that the level of trainers’ perception has a significant effect on ICT integration into teaching and learning. It was concluded that trainers are willing to undergo training to equip them with new knowledge in ICT to improve their productivity. It was recommended that the National Polytechnic management, in partnership with the Ministry of Education, should carry out assessment of skills and needs as far as ICT integration is concerned.

: Teaching nowadays is fast expanding rapidly and it requires modern technologies to provide more flexible and effective ways of improving teacher education, and connect ing teachers to the global community. ICT ( Information and Communication Technology) integration into teacher education , therefore, is the key to equipping and producing professional teachers as well as improved pedagogy. However, there is the need to consider the type and nature of teacher trainees before integrating ICT in teacher education programmes, because their perceptions are likely to differ. This study , therefore , focuses on the perceptions of teacher trainees (regular and sandwich students) regarding the integration of ICT in teacher education programme at the UNILAG ( University of Lagos ) in Akoka, Nigeria. It adopted the descriptive research design. Two hundred and fifty participants were involved. One research question and two hypotheses, based on the specific purposes , were raised and answered in the study. A self-constructed validated and reliable instrument was used to collect data. Results revealed that teacher trainees are favourably disposed to the integration of ICT into teacher education as indicated by the grand mean and standard deviation (M = 3.22; SD = 12.22). Significant differences in the perceptions of teacher-trainees regarding the integration of ICT in teacher education programme based on type of the programme was also discovered (t-cal = -2.315; df = 98; P < .05). Teacher-trainees have to realise that ICT has come to stay in their programme. They need to develop more confidence in their individual ability at becoming computer literate teachers in order to remain relevant in the modern day information age. KEY WORDS: Teacher Education ; Information and Communication Technology ; Teacher-Trainees ; Perceptions ; C omputer L iterate T eachers. About the Authors: Dr. Olusola Angelina Thomas is an Academic Staff at the Department of Educational Administration UNILAG (University of Lagos) in Akoka, Nigeria; and Dr. Maruff Akinwale Oladejo is currently a Lecturer at the Department of Educational Administration UNILAG in Akoka, Nigeria. For academic interests, the authors can be contacted via their e-mails at: soowler@yahoo.com and maoladejo@unilag.edu.ng Suggested Citation: Thomas, Olusola Angelina & Maruff Akinwale Oladejo. (2017). “Teacher-Trainees’ Perceptions of ICT (Information Communication Technology) Integration in Nigerian Teacher Education Programme” in EDUCARE: International Journal for Educational Studies , Volume 10(1), August, pp.1-10. Bandung, Indonesia: Minda Masagi Press owned by ASPENSI, ISSN 1979-7877. A rticle Timeline : Accepted (February 22, 2017); Revised (May 20, 2017); and Published (August 30, 2017).

Reading comprehension is one of the most critical abilities in human intelligence, and machine reading comprehension is one of the most popular research directions in the field of natural language processing. Machine reading is one of the key aspects to achieve artificial intelligence. This paper first analyzes the current state of development of machine reading comprehension, and based on the current problems in machine reading comprehension, this paper applies various attention mechanisms in the reading comprehension model, so that the information of articles and questions can be fused with each other to extract the key information and predict the answers more accurately. This paper also designs experiments and verifies that this method has a higher accuracy rate compared with other methods.

One of the emerging research trends in natural language understanding is machine reading comprehension (MRC) which is the task to find answers to human questions based on textual data. While many datasets have been developed for MRC research for other languages, there is a lack of such resources for the Vietnamese language. Although many datasets and methodologies have been developed for English and Chinese, many Vietnamese machine reading comprehension limitations need to be solved further. Existing Vietnamese datasets for MRC research concentrate solely on answerable questions. However, in reality, questions can be unanswerable for which the correct answer is not stated in the given textual data. To address the weakness, we provide the research community with a benchmark dataset named UIT-ViQuAD 2.0 for evaluating the MRC task and question answering systems for the Vietnamese language. We use UIT-ViQuAD 2.0 as a benchmark dataset for the shared task on Vietnamese machine reading comprehension (VLSP2021-MRC) at the Eighth Workshop on Vietnamese Language and Speech Processing (VLSP 2021). This task attracted 77 participant teams from 34 universities and other organizations. Each participant was provided with the training data, including 28,457 annotated question-answer pairs, and returned the result on a public test set of more than 3,821 questions and a private test set of 3,712 questions. In this article, we present details of the organization of the shared task, an overview of the methods employed by shared-task participants, and the results. The highest performances in this competition are 77.24% (in EM) and 67.43% (in F1-score) on the private test set. The Vietnamese MRC systems proposed by the top 3 teams use XLM-RoBERTa, a powerful pre-trained language model based on the transformer architecture that has achieved state-of-the-art results on many natural language processing tasks. We believe that releasing the UIT-ViQuAD 2.0 dataset motivates more researchers to improve Vietnamese machine reading comprehension.

As a broad and major category in machine reading comprehension (MRC), the generalized goal of discriminative MRC is answer prediction from the given materials. However, the focuses of various discriminative MRC tasks may be diverse enough: multi-choice MRC requires model to highlight and integrate all potential critical evidence globally; while extractive MRC focuses on higher local boundary preciseness for answer extraction. Among previous works, there lacks a unified design with pertinence for the overall discriminative MRC tasks. To fill in above gap, we propose a lightweight POS-Enhanced Iterative Co-Attention Network (POI-Net) as the first attempt of unified modeling with pertinence, to handle diverse discriminative MRC tasks synchronously. Nearly without introducing more parameters, our lite unified design brings model significant improvement with both encoder and decoder components. The evaluation results on four discriminative MRC benchmarks consistently indicate the general effectiveness and applicability of our model, and the code is available at https://github.com/Yilin1111/poi-net.

Despite the remarkable progress on Machine Reading Comprehension (MRC) with the help of open-source datasets, recent studies indicate that most of the current MRC systems unfortunately suffer from weak robustness against adversarial samples. To address this issue, we attempt to take sentence syntax as the leverage in the answer predicting process which previously only takes account of phrase-level semantics. Furthermore, to better utilize the sentence syntax and improve the robustness, we propose a Syntactic Leveraging Network, which is designed to deal with adversarial samples by exploiting the syntactic elements of a question. The experiment results indicate that our method is promising for improving the generalization and robustness of MRC models against the influence of adversarial samples, with performance well-maintained.

Machine Reading Comprehension (MRC) is the process where computers or, machines are taught to understand a paragraph or more technically called a context. Like humans, machines also need to be evaluated for their understanding on question answering. MRC is one of the formidable sub-domains in the Natural Language Processing (NLP) domain, which has seen considerable progress over the years. In recent years, many novel datasets have tried to challenge the Machine Reading Comprehension (MRC) models with inference based question answering. With the advancement in NLP, many models have surpassed human-level performance on these datasets, albeit ignoring the obvious disparity between genuine human-level performance and state-of-the-art performance. This highlights the need for attention on the collective improvement of existing datasets, metrics, and models towards “real” prehension. Addressing the lack of sanity in the domain, this paper performs a comparative study on various transformer based models and tries to highlight the success factors of each model. Subsequently, we discuss an MRC model that performs comparatively better, if not the best, on question answering and give directions for future research.KeywordsTransformerNatural language processingReading comprehensionQuestion answeringDeep learning

BackgroundExtractive methods for machine reading comprehension (MRC) tasks have achieved comparable or better accuracy than human performance on benchmark data sets. However, such models are not as successful when adapted to complex domains such as health care. One of the main reasons is that the context that the MRC model needs to process when operating in a complex domain can be much larger compared with an average open-domain context. This causes the MRC model to make less accurate and slower predictions. A potential solution to this problem is to reduce the input context of the MRC model by extracting only the necessary parts from the original context.ObjectiveThis study aims to develop a method for extracting useful contexts from long articles as an additional component to the question answering task, enabling the MRC model to work more efficiently and accurately.MethodsExisting approaches to context extraction in MRC are based on sentence selection strategies, in which the models are trained to find the sentences containing the answer. We found that using only the sentences containing the answer was insufficient for the MRC model to predict correctly. We conducted a series of empirical studies and observed a strong relationship between the usefulness of the context and the confidence score output of the MRC model. Our investigation showed that a precise input context can boost the prediction correctness of the MRC and greatly reduce inference time. We proposed a method to estimate the utility of each sentence in a context in answering the question and then extract a new, shorter context according to these estimations. We generated a data set to train 2 models for estimating sentence utility, based on which we selected more precise contexts that improved the MRC model’s performance.ResultsWe demonstrated our approach on the Question Answering Data Set for COVID-19 and Biomedical Semantic Indexing and Question Answering data sets and showed that the approach benefits the downstream MRC model. First, the method substantially reduced the inference time of the entire question answering system by 6 to 7 times. Second, our approach helped the MRC model predict the answer more correctly compared with using the original context (F1-score increased from 0.724 to 0.744 for the Question Answering Data Set for COVID-19 and from 0.651 to 0.704 for the Biomedical Semantic Indexing and Question Answering). We also found a potential problem where extractive transformer MRC models predict poorly despite being given a more precise context in some cases.ConclusionsThe proposed context extraction method allows the MRC model to achieve improved prediction correctness and a significantly reduced MRC inference time. This approach works technically with any MRC model and has potential in tasks involving processing long texts.

Machine reading comprehension (MRC) is a challenging natural language processing (NLP) task. It has a wide application potential in the fields of question answering robots, human‐computer interactions in mobile virtual reality systems, etc. Recently, the emergence of pretrained models (PTMs) has brought this research field into a new era, in which the training objective plays a key role. The masked language model (MLM) is a self‐supervised training objective widely used in various PTMs. With the development of training objectives, many variants of MLM have been proposed, such as whole word masking, entity masking, phrase masking, and span masking. In different MLMs, the length of the masked tokens is different. Similarly, in different machine reading comprehension tasks, the length of the answer is also different, and the answer is often a word, phrase, or sentence. Thus, in MRC tasks with different answer lengths, whether the length of MLM is related to performance is a question worth studying. If this hypothesis is true, it can guide us on how to pretrain the MLM with a relatively suitable mask length distribution for MRC tasks. In this paper, we try to uncover how much of MLM’s success in the machine reading comprehension tasks comes from the correlation between masking length distribution and answer length in the MRC dataset. In order to address this issue, herein, (1) we propose four MRC tasks with different answer length distributions, namely, the short span extraction task, long span extraction task, short multiple‐choice cloze task, and long multiple‐choice cloze task; (2) four Chinese MRC datasets are created for these tasks; (3) we also have pretrained four masked language models according to the answer length distributions of these datasets; and (4) ablation experiments are conducted on the datasets to verify our hypothesis. The experimental results demonstrate that our hypothesis is true. On four different machine reading comprehension datasets, the performance of the model with correlation length distribution surpasses the model without correlation.

Dialogue state tracking (DST) is at the heart of task-oriented dialogue systems. However, the scarcity of labeled data is an obstacle to building accurate and robust state tracking systems that work across a variety of domains. Existing approaches generally require some dialogue data with state information and their ability to generalize to unknown domains is limited. In this paper, we propose using machine reading comprehension (RC) in state tracking from two perspectives: model architectures and datasets. We divide the slot types in dialogue state into categorical or extractive to borrow the advantages from both multiple-choice and span-based reading comprehension models. Our method achieves near the current state-of-the-art in joint goal accuracy on MultiWOZ 2.1 given full training data. More importantly, by leveraging machine reading comprehension datasets, our method outperforms the existing approaches by many a large margin in few-shot scenarios when the availability of in-domain data is limited. Lastly, even without any state tracking data, i.e., zero-shot scenario, our proposed approach achieves greater than 90% average slot accuracy in 12 out of 30 slots in MultiWOZ 2.1.

This paper presents the development process of a Vietnamese spoken language corpus for machine reading comprehension (MRC) tasks and provides insights into the challenges and opportunities associated with using realworld data for machine reading comprehension tasks.The existing MRC corpora in Vietnamese mainly focus on formal written documents such as Wikipedia articles, online newspapers, or textbooks.In contrast, the VlogQA consists of 10,076 question-answer pairs based on 1,230 transcript documents sourced from YouTube -an extensive source of user-uploaded content, covering the topics of food and travel.By capturing the spoken language of native Vietnamese speakers in natural settings, an obscure corner overlooked in Vietnamese research, the corpus provides a valuable resource for future research in reading comprehension tasks for the Vietnamese language.Regarding performance evaluation, our deep-learning models achieved the highest F1 score of 75.34% on the test set, indicating significant progress in machine reading comprehension for Vietnamese spoken language data.In terms of EM, the highest score we accomplished is 53.97%, which reflects the challenge in processing spoken-based content and highlights the need for further improvement.

We propose a simple and effective strategy for data augmentation for low-resource machine reading comprehension (MRC). Our approach first pretrains the answer extraction components of a MRC system on the augmented data that contains approximate context of the correct answers, before training it on the exact answer spans. The approximate context helps the QA method components in narrowing the location of the answers. We demonstrate that our simple strategy substantially improves both document retrieval and answer extraction performance by providing larger context of the answers and additional training data. In particular, our method significantly improves the performance of BERT based retriever (15.12%), and answer extractor (4.33% F1) on TechQA, a complex, low-resource MRC task. Further, our data augmentation strategy yields significant improvements of up to 3.9% exact match (EM) and 2.7% F1 for answer extraction on PolicyQA, another practical but moderate sized QA dataset that also contains long answer spans.

Machine reading comprehension (MRC) has recently reached human-level accuracy on resource-rich languages (e.g. English). However, for low-resource MRC, there is a huge gap between machine and human performance due to limited annotated data. To narrow this gap, we investigate three strategies, namely data augmentation via translation, multilingual training and cross-lingual fine-tuning, to improve low-resource MRC via knowledge transfer. Experiments on a small Chinese MRC dataset (CMRC2018), demonstrate that our strategies are capable of leveraging knowledge from the English SQuAD dataset. Furthermore, the combination of the three strategies achieves significant improvements on the DRCD (the Delta Reading Comprehension Dataset).

Chinese electronic medical records named entity recognition (NER) is a core task in medical knowledge mining, which is usually viewed as a sequence labeling problem. Recent works introduce the machine reading comprehension (MRC) framework into this task and extract named entities in a question-answering manner, resulting in state-of-the-art performance. However, they extract entities of different types independently, ignoring the fact that entities presented in the context might highly correlate with each other. To address this issue, we extend the MRC-based model and propose Fusion Label Relations with MRC (FLR-MRC). The method implicitly models the relations between labels through graph attention networks and fuse label information with text for named entity recognition. Experimental results on the benchmark datasets CMeEE and CCKS2017-CNER demonstrate FLR-MRC outperform existing clinical NER methods, with F1-score reaching 0.6652 and 0.9101, respectively.KeywordsMedical named entity recognitionLabel relationsMachine reading comprehension

Machine Reading Comprehension (MRC) is a challenging task in Natural Language Processing (NLP) field. It is a fast evolving field in the recent years because of the development of large-scale datasets and the deep learning technology. Even though, numerous MRC models have been developed for Question Answer (QA) system, the existing MRC models are not on par with reading comprehension. In order to assess and improve the performance of the projected datasets, an advanced level of evaluation metrics have been introduced. Therefore, this work has conducted a survey over the existing MRC models and evaluation metrics utilized in them to address the lack of MRC with human reading.